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Preface to the First Edition 


y love of photography started 
very early. As a teenager I 
worked in a small, full-service 


photography shop. Portraits were made, 
events were photographed, snapshots were 
developed and printed, and equipment was 
sold. From that experience, I learned about 
photographic techniques and the value of 
quality cameras and lenses. I started devel- 
oping and enlarging my own photographs, 
and I searched for ways to learn more. 
Fortunately, I found the book Lenses in 
Photography by Rudolf Kingslake in the pho- 
tography shop, and I studied it diligently. I 
still have that book and refer to it frequently. 
At that time (the 1950s) my uncle was serv- 
ing with the Air Force in Germany, and he 
was able to buy fine cameras for me. First, 
I got a Zeiss Ikonta 35. It was bare bones, 
with no rangefinder or light meter, but it 
had a wonderful Zeiss Tessar f/2.8 lens. It 
was great for documenting sports and other 
high school events. Later, when I started 


doing freelance photography (while still 
in high school), I ordered a Rolleicord III 
medium format camera. It served me well 
and is still functional. 

From the beginning I was fascinated by 
all aspects of photography. I love the equip- 
ment, the techniques, the processing of 
images, and, of course, the chance to pho- 
tograph interesting things. Photography 
also provided a summer job and a door- 
way to business and social interactions. 
Photographs documenting those years 
reveal small-town life in the 1950s and a few 
cheesecake pictures as well. My career as a 
scientist and a university professor took me 
away from photography for many years, but 
in the past decade I have returned to that 
early love. I spend a lot of time on nature 
photography now, and I enjoy photo shoots 
with the Carolinas’ Nature Photographers 
Association (CNPA). Of course, everything 
is digital so the chemical darkroom is no 
longer necessary. This has given me a new 
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world of opportunities and an array of new 
subjects to understand. 

In my case, understanding the way pho- 
tography works has increased my enjoyment 
of it. Each new question is a challenge. The 
process of working through the concepts of 
photography, from basic optics and image 
sensors to human perception of color and 
the appreciation of beauty, was an exhilarat- 
ing experience for me. I have written this 
book for those who also enjoy photography 
and who want to know more about their 
photographic equipment, and the operation 
of their eyes and brain as well. The book is 
specifically aimed at those who enjoy sci- 
ence and are not afraid of a little math. Of 
course, perfectly good photographs can be 
made by those who have no interest in the 
scientific side of photography. They see a 
clean separation between the scientific part 
and the artistic part, and they reject the sci- 
entific part. On the other hand, some great 


photographers and other artists as well have 
benefited from a knowledge of their media 
and ways to get the most out of it; Ansel 
Adams comes to mind. In addition to mak- 
ing awe-inspiring photographs, he wrote 
technical books on cameras, negatives, 
and prints. To each his or her own, but I 
believe that in photography, as elsewhere, 
knowledge is power. 

I have worked on this book for four years, 
trying the patience of my wife and friends. 
I appreciate comments from all those who 
have read sections of it at various stages of 
its gestation. I am sure to leave out some 
generous and helpful people, but here is at 
least a partial list of those who have con- 
tributed at various times with corrections 
and advice: John Fowler, Archibald Fripp, 
Richard Jarnagin, and Calvin Wong. 


CHARLES S. JOHNSON, JR. 
CHAPEL HILL, 2009 


Preface to the Second Edition 


been updated to reflect advances in 

camera design and imaging technology. 
Also, more attention is paid to the practi- 
cal aspects of digital photography, such as 
camera settings, image formats, and photo 
processing, and the sections on panoramas 
and UV/IR capture have been expanded. 

A chapter concerning how we see and 
how we image nature has been added to 


E the second edition, sections have 


extend the discussion of human percep- 
tion and the eye/brain system. There are 
also new appendices that address color 
management in the presence of color vision 
anomalies and the derivation of depth-of- 
field equations. 

I would like to thank Peter Finkelstein 
for his comments and corrections. 
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CHAPTER 1 


What Is Photography? 


The painter constructs, 
the photographer discloses. 


—SUSAN SONTAG 


You don’t take a photograph, 
you make it. 


—ANSEL ADAMS 


T™ are two sides to photography. 
First, photography is the capture and 
display of images by means of film 
or an electronic sensor; and, second, pho- 
tography is the art of taking and presenting 
photographs. As commonly practiced, pho- 
tography is inseparable from cameras. Of 
course, photography means “writing with 
light” and writing is really the operative 
word. When photography was discovered in 
1839, the thing that was discovered was the 
means for permanently capturing images. 
Cameras of various kinds had, in fact, been 
available for centuries. 

The original camera, known as the cam- 
era obscura (see Figure 1.1), was nothing 
more than a dark room with a small hole 
(aperture) in one wall and an inverted image 
on the opposite wall created by light rays 
passing through the aperture. The wonderful 
image forming property of a small aperture 
was noted by the philosophers Mo-Ti in 
China and Aristotle in Greece in the 5th and 


4th centuries BCE, respectively, although 
apparently without an understanding of how 
it was accomplished. It is clear that scientists 
in the Western world from at least the time 
of Leonardo da Vinci (c. 1490) were aware 
of the camera obscura, and at some point it 
was discovered that the image quality and 
intensity could be improved by enlarging the 
aperture and inserting a convex lens of the 
appropriate focal length. The portable cam- 
era obscura, a box with a lens on one side and 
some means of viewing the image, became 
popular with artists as an aid in represent- 
ing perspective in paintings. For example, 
the 16th-century Dutch painter Johannes 
Vermeer (1632-1675) almost certainly used 
a camera obscura to see the correct represen- 
tation of perspective for his paintings. By the 
19th century these devices were essentially 
box cameras without photographic film. 

In the early 19th century many indi- 
viduals were experimenting with sensitized 
materials that would darken when exposed 
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FIGURE 1.1. The camera obscura was used by Reinerus Gemma-Frisius in 1544 to observe an 
eclipse of the sun. 


to light to produce fleetingimages, so proper 8 hour exposure on bitumen-coated pewter. 
credit for the invention of photography is However, photography, as we know it, dates 
diffuse and controversial. Probably the first from 1839, when two men independently 
permanent image from nature was obtained reported processes for capturing images 
by Nicéphore Niépce in 1926/1927 withan in the camera obscura. Their disclosures 
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FIGURE 1.2. Louis Mandé Daguerre (left) and Henry Fox Talbot (right). 


initiated explosive developments in image 
making around the world. A Frenchman, 
Louis Mandé Daguerre, discovered a 
method for producing a permanent image 
ona silver surface; while, in England, Henry 
Fox Talbot created permanent images on 
paper treated with a mixture containing sil- 
ver chloride. In Daguerre’s images, the areas 
exposed to light and properly processed 
were highly reflecting; and, therefore, there 
was a natural (positive) appearance, though, 
of course, without color (monochrome). 
These images, which were called daguer- 
rotypes, unfortunately could not easily be 
reproduced. However, the striking images 
obtained by Daguerre were an instant hit, 
and most contemporaries considered him 
to be the inventor of photography. (See 
Figure 1.2.) 

In marked contrast, Talbot’s images were 
initially unpleasing because the (bright) 
exposed areas were found to be dark on 
the paper. In other words, a negative image 
was produced. However, that turned out 
to be a great advantage because the nega- 
tive could be combined with another sheet 
of sensitized paper and exposed to light to 
produce a positive copy, and that procedure 
could be repeated to produce multiple cop- 
ies. Of course, paper is translucent rather 
than transparent, and it was not until the 
1850s that transparent negatives could be 
obtained. 

Apparently, the astronomer Johann 
Heinrich von Madler coined the term 
photography in 1839 when he combined 
the Greek words for “light” and “to write” 
However, the terms photography and 
photograph are usually attributed to Sir 
John Herschel, who included them in a 
paper that he read to the Royal Society of 
London in 1839. Herschel deserves credit 
for advancing photographic science by 
discovering how to stabilize silver images. 
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The term photography may actually have 
been introduced earlier by an artist named 
Hercules Florence working in Brazil in 1833. 
Florence, who used sensitized paper to copy 
drawings, did not report his work, and, as a 
consequence, he had little influence on the 
development of photography. 

For the next 160 years, silver sensitized 
paper and film coupled with the negative/ 
positive process dominated photography, 
and it was only after the year 2000 that 
photoelectric sensors and powerful, yet inex- 
pensive, computers challenged film-based 
photography. Replacing film with sensors 
and computer memory has not yet basically 
changed photography; however, computer 
manipulation ofimages has turned out to be 
a revolutionary development. Even the few 
images still captured on film are now rou- 
tinely scanned into computers and digitized 
so that they are also subject to modifica- 
tion and easy reproduction and printing. If 
computer image manipulations were limited 
to the types of things that photographers 
were already doing in the darkroom to cor- 
rect exposure, hold back or burn in areas, 
change contrast, etc., there would be no fun- 
damental change in photography. But now 
the changes can be so extensive and sub- 
tle that the boundaries of photography are 
continuously being tested. 

It has been said that photographers 
reveal, while artists create. Software for 
manipulating photographic images and 
even creating realistic images from scratch 
is fundamentally changing this equation. 
Illustrators, using computer graphics, have 
almost unlimited ability to produce real- 
istic images of any type. Photojournalists, 
on the other hand, must have their crea- 
tive inclinations severely limited by a 
code of professional ethics and perhaps by 
authentication software that can spot even 
microscopic changes in digital images. 
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Anyone who has viewed recent movies 
knows that amazing things can be done to 
produce realistic images of things that never 
existed. The opening scenes in The Day 
After Tomorrow show a flight over ocean, 
ice, and cliffs in Antarctica. It is beautiful 
and impressive. How was it done? A heli- 
copter flight over those remote areas would 
be costly and dangerous, so the producer 
decided to create the scenes entirely with 
computer graphics. And what about the 
magnificent scenes in the epic Troy? Does 
anyone believe that 1000 or even 100 ships 
were constructed, or that 75,000 Greek 
warriors took part in the battle? Computer- 
generated imagery (CGI) has certainly come 
of age as we can enjoy 3D spectacles such 
as the movie The Life of Pi, mostly created 
by Rhythm & Hues Studios, and wonder 
which scenes were actually photographed 
in nature. We may enjoy the endless pos- 
sibilities for image making, but we can no 
longer (if ever we did) believe in what the 
images show. 

So, ultimately, what is photography? 
Does it matter that wrinkles can be removed 
from faces and heads can even be switched? 
Do we care if it is easy to move an alligator 
from a zoo to a natural area or a humming- 
bird from a feeder to a flower? Some gullible 
observers may still marvel at how such “dif- 
ficult” photographs could be obtained. We 
are seeing the emergence of a new art form, 
but I am not sure where that leaves pho- 
tography. Will “pure” photography remain 
when everyone has an incentive to improve 
the images they obtain and it is so easy to 
do even in camera? Is there any merit in 
maintaining photography with minimum 
manipulation for recording the world as 
it is? The future will tell. In fact, “truth” is 
found in some novels and paintings and in 


some photographs, but it must be tested 
and verified by wise readers and observers. 

Questions raised by the concepts of 
reality and truth in visual images are much 
more complex than may be thought. Later 
in the book I discuss the operation of the 
human visual system and its relation to our 
awareness of the world. It is fair to say that 
our eyes and brain create the illusion of a 
full-color, three-dimensional world. It is 
an illusion because the images projected 
on the retinas of our eyes do not provide 
enough information for the construction 
of a unique world view. The brain fills in 
details based on a sort of automatic infer- 
ence system that is influenced by both the 
evolution of the human brain and the expe- 
rience of the individual. The result is that 
we see, at first glance, pretty much what we 
expect to see. One should also realize that 
digital cameras basically compute pictures 
from captured light. The computation is not 
straightforward, and there is a lot of room 
for “enhancement” of the final image. The 
new field of computational photography 
is influencing the images produced by our 
cameras and the special effects we see in 
movies. It is an interesting time to be alive 
and maybe a little disturbing as well. 


Further Reading 


Peres, M. R., Editor (2007). FOCAL 
Encyclopedia of Photography. Amsterdam: 
Elsevier. An extensive coverage of theory, 
applications, and science, but uneven and 
already dated. 

Rosenblum, N. (1997). A World History 
of Photography, 3rd Ed. New York: 
Abbeville Press. A tour of photography 
from 1839 through the film era, includ- 
ing both art and the technical details. 


What Is Light? 


All the fifty years of conscious brooding have brought me no 
closer to the answer to the question, “What are light quanta?” 
Of course today every rascal thinks he knows the answer, 
but he is deluding himself. 

—ALBERT EINSTEIN, 1951 


| therefore take the liberty of proposing for this hypothetical 
new atom, which . . . plays an essential part in every 
process of radiation the name photon. 
—G.N. LEWIS IN NATURE, DECEMBER 18, 1926 


e can wax eloquent: Light is uni- 
versally revered. It represents 
illumination and understand- 


ing. It represents our greatest treasure, as 
in the “light of our lives” We believe that 
bad things happen under cover of darkness, 
and evil works cannot stand the light of day. 
Light sets the pattern of our days. When 
light goes away, we sleep. Photography 
involves writing with light, and it uses the 
information in light to capture moments in 
our lives so that they are preserved and can 
be replayed. In this way, photography brings 
light to our past. 

But here we need useful rather than 
poetic definitions that can help us under- 
stand photography. We all know the 
difference between light and dark, but what 


is light? The problem is that most defini- 
tions introduce other words that need to 
be defined. I will start with several ideas 
that are correct but not complete as a step 
toward understanding what we mean by 
light. First, light is a form of energy that 
comes to us from some source and can 
make our environment visible to us. Of 
course, even with our eyes closed we can 
tell the difference between bright light and 
no light. Another useful statement is that 
light is a kind of radiation that is visible to 
us. Again this implies energy, the ability 
to do work and affect change. 

All of our energy, other than that associ- 
ated with radioactivity, comes to us directly 
or indirectly from the sun, and the sun is 
our major source of light. The sun provides 
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FIGURE 2.1. The sun, our primary source of light and energy. 


bright “white” light in the middle of the day 
and less intense yellow and red shades in the 
early morning and late afternoon, at least as 
seen from the surface of the earth (Figure 
2.1). This sentence brings up the concepts 
of intensity and color, both of which are 
related to human perceptions. Therefore, 
the study of light in photography requires 
not only precise mathematical definitions of 
brightness and color, but also methods of 
quantifying perceptions of brightness and 
color. The latter gets us into the realm of 
psychophysics. 

Later chapters will be devoted to the 
measurement of light, the manipulation of 
light, and the recording of light. Here I con- 
sider only the primary physical properties of 
light, starting with a bit of history. In the 4th 
and 5th centuries BCE, the ancient Greeks 
speculated extensively about the nature of 
light. Some (Empedocles, Euclid, and Plato) 
thought that light is projected from the eye. 
Lucretius believed that particles of light are 
sent out from the sun, while Pythagoras 
held that light particles are emitted by 
objects. Aristotle even offered a theory of 
wave propagation of light. Some of these 
ideas have elements of truth, but they were 
not particularly useful and did not advance 
knowledge because they were not based 
on a consistent theoretical framework and 
were not tested. In fact, these ideas only 
provided a set of possibilities. 


There were, of course, some observations 
and insights in the ancient world that carry 
over into the scientific study of light. For 
example, Euclid (4th and 3rd centuries BCE) 
noted that light appears to travel in a straight 
line, and Hero of Alexandria (1st century 
BCE) concluded from the study of reflections 
that light follows the shortest path between 
two points. However, modern ideas about 
the nature of light and optics have their ori- 
gin in the 17th century. At that time, two 
ideas dominated theory and experiment: (1) 
light is a propagating wave and (2) light con- 
sists of streams of particles (corpuscles). 

One of the most influential scientists of 
that period was Sir Isaac Newton (1642— 
1727; see Figure 2.2), who had carried out 
exquisite optical experiments to determine 
the nature of color and who developed a 
theory of color we still use. He concluded 
that light rays consist of particles rather 
than waves because light travels in a straight 
line. Anyone who has played with a laser 
pointer might reach the same conclusion. 
Now we know that this view is incomplete 
because, among other things, shadows do 
not have sharp edges, and there is much evi- 
dence of diffraction in the everyday world. 
I will treat diffraction in detail later; here it 
suffices to say that diffraction refers to the 
deflection and decomposition that occurs 
when light interacts with sharp edges of 
opaque objects, narrow slits, or collections 
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FIGURE 2.2. Sir Isaac Newton (1642-1727), who dispersed light with a prism and developed a 


theory of color. 


of narrow slits (gratings). In fact, in 1663 
James Gregory noticed that sunlight pass- 
ing through a feather is diffracted into spots 
of different colors, and he wrote, “I would 
gladly hear Mr. Newton’s thoughts of it” 
Francesco Grimaldi even published a book 
in 1665 on the effects of small apertures 
on the passage of light beams that seemed 
to indicate wave-like properties for light. 
Apparently, Newton did not consider the 
possibility that light might be a wave with 
a very small wavelength so that it would 
indeed appear to travel in a straight line 
unless one looked very closely. 

The particle theory was, in fact, under- 
mined by continuing observations of 
diffraction and the presentation of a wave 
theory of light by Christiaan Huygens in 
1678 that explained such phenomena, 
albeit with certain unproved assumptions. 


According to the wave theory, diffracted 
intensity patterns, which consist of bright 
and dark lines, result from interference 
effects, that is to say, the addition of the 
amplitudes of waves. In this way the posi- 
tive and negative peaks can reinforce to give 
intensity maxima or cancel to produce the 
dark places. This process is illustrated in 
Figure 2.3. The validity of the wave theory 
was confirmed by a number of scientists, 
most notably Thomas Young (1802), who 
demonstrated the interference (diffraction) 
of light by passing a light beam through two 
closely spaced slits in an opaque sheet and 
displaying the resulting pattern of bright 
and dark lines. Also, Joseph Fraunhofer and 
Augustine Fresnel investigated diffraction 
effects by small holes of different shapes 
and presented rigorous theories for calcu- 
lating diffraction patterns, thus advancing 
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Destructive 
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FIGURE 2.3. The constructive (a) and destructive (b) interference of waves. 


the wave hypothesis. The diffraction of light 
from a compact disc (Figure 2.4) where 
there are 625 tracks per millimeter provides 
a beautiful example. 

By the early 19th century the wave prop- 
erties of light were well-known, and there 
was a practical understanding of the way 
light rays behave and of the basis of perceived 
color resulting from the mixing of light 
beams. However, there was still the open 
question, “What is light?” One line of evi- 
dence was provided by the speed of light. To 
casual observers, the speed of light appears 


to be infinite, but well-planned experiments 
show that is not the case. An early, approxi- 
mate value of the speed of light had been 
determined from observations of eclipses of 
the moons of Jupiter by Olaf Roemer (1676), 
and by 1860 there were measurements of 
the speed of light in air from the study of 
light pulses (Armand Fizeau) and rotating 
mirrors (J. L. Foucault). The conclusion of 
this part of the story was provided by James 
Clerk Maxwell’s monumental achievements 
in the 1860s. Maxwell (see Figure 2.5) devel- 
oped a set of partial differential equations 


FIGURE 2.4. Diffraction of light from the surface of a compact disc. The pitch of the data tracks 
is 1.6 um, and each angle of reflection selects a specific spectral color. 
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FIGURE 2.5. James Clerk Maxwell (1831-1879), who identified light as electromagnetic radiation. 


to describe electric and magnetic fields. He 
concluded that oscillating electric fields are 
always accompanied by oscillating magnetic 
fields, and together they propagate through 
space as electromagnetic waves. In 1862 
he computed the velocity of electromag- 
netic waves and discovered that they have 
the same velocity as light. This revelation 
prompted him to write in an 1864 paper, 
“we have reason to believe that light itself 
(including radiant heat and other radia- 
tions) is an electromagnetic disturbance in 
the form of waves propagated through the 
electromagnetic field according to electro- 
magnetic laws.” So the simple answer to 
the question “What is light?” is that light is 
electromagnetic (EM) radiation. 

But that was not the end of the story. In 
1905 Albert Einstein (see Figure 2.6) showed 
that light is composed of quanta (particles 
of energy), and every experiment since that 
time confirms his conclusion. What he had 


discovered was that light quanta are neces- 
sary to explain the interaction of light with 
electrons in metals. For this amazing con- 
ceptual breakthrough Einstein was awarded 
the Nobel Prize in 1921. Newton was right, 
but for the wrong reasons. We always detect 
light quanta, now called photons; but the 
wave theory permits us to compute the 
probability of finding a photon at a certain 
location. Each photon carries an amount of 
energy that depends on the frequency of the 
light, and the number of photons arriving 
each second determines the intensity. A few 
photons are required to activate a nerve in 
the eye, and the creation of a latent image in 
photographic film requires the absorption 
of photons by atoms in silver halide grains. 
Now there are many instruments that can 
detect single photons. 

So we have a beautiful and subtle story. 
Light is made of particles and light is a 
wave. Maxwell’s equations are sufficient for 
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FIGURE 2.6. Albert Einstein (1879-1955), who discovered that light is quantized. 


describing light when there are many pho- 
tons, and the quantum theory is necessary 
when there are only a few. No wonder that 
the ancients had such a hard time getting it 
straight. 

In later chapters we will explore in 
detail how the nature of light is manifest in 
photography and especially digital photog- 
raphy. We will see how the wave nature of 
light ultimately determines the way lenses 
work but also limits the resolution of all 
lenses. For example, the size of the image of 
a distant star is determined by diffraction, a 
wave phenomenon, as well as by imperfec- 
tions in the lens and the resolving power of 
film or an electronic sensor. Also, detection 


and recording of an image depends on the 
collection of photons in each picture ele- 
ment (pixel) of the sensor or by grains of 
silver halide in a photographic film. When 
the light level is low, the number of pho- 
tons in each pixel may be small so that 
deviations from the average number will be 
significant. Therefore, different pixels will 
receive different numbers of photons and 
the image will appear to be “noisy” We shall 
see that the laws of physics ultimately limit 
the quality of images and force us to make 
compromises in camera and lens designs. I 
expand on this topic and define the signal- 
to-noise ratio in Chapter 16. 


CHAPTER 3 


The Camera—An Introduction 


No photographer is as good as the simplest camera. 
—EDWARD STEICHEN 


The camera is an instrument that teaches 
people how to see without a camera. 


—DOROTHEA LANGE 


3.1 Introduction 


uch of this book is concerned 

with the components of pho- 

tographic cameras and the way 
they operate. This chapter outlines the major 
features of cameras as an introduction to 
the important topics that will be treated in 
more detail in later chapters. Also, I will take 
this opportunity to define the parameters 
that characterize lenses, films, and sensors. 
In later chapters I consider the resolving 
power of lenses, the sensitivity of sensors, 
the effects of diffraction, etc. An impor- 
tant goal of this part of our study will be to 
show under what conditions an “equivalent 
image” can be obtained with cameras having 
film/sensors of different sizes. 

A camera is essentially a box (light-proof 
housing) that contains a lens and a light sen- 
sor. We also expect the camera to include 
an aperture to control the amount of light 


passing through the lens, a shutter to con- 
trol the duration of the exposure, and some 
kind of viewfinder to frame the object to 
be photographed. The earliest cameras 
were quite large, and the light detector was 
a sensitized photographic plate. The plate 
was exposed and then developed to make a 
negative, and the negative was used to make 
a positive contact print of exactly the same 
size. Since there was usually no enlarge- 
ment, the lenses were not severely tested 
and the resulting prints often appeared to 
be of good quality. Of course, some photo- 
graphic processes, e.g. daguerreotype and 
tintype, directly produced positive images 
on metal plates. (It is interesting to note 
that these images were reversed left to 
right so that, for example, in a famous Civil 
War portrait of President Lincoln his mole 
appeared on the wrong cheek. This is not a 
problem with prints made from negatives 
since they are reversed twice.) 
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3.2 The Camera Body 


way to make a larger photograph was 

to use a larger camera. In spite of suc- 
cessful efforts to develop enlargers for 
photographs, many professional photog- 
raphers preferred large plate film cameras. 
It is reported that William Henry Jackson 
hauled a 20 in. x 24 in. inch glass plate cam- 
era on mule back to photograph the western 
mountains. Also, Ansel Adams (1902-1984) 
often used a camera with 8 in. x 10 in. film 
plates. But certainly George R. Lawrence 
takes the cake. In 1901 he photographed 
a train with a camera that could take pho- 
tographs measuring 4 ft. 8 in. x 8 ft. That 
gives a whole new meaning to large format 
photography. In contrast to this, most pho- 
tographers clamored for portable cameras 
with small negatives that could be enlarged. 
An early example is shown in Figure 3.1. 
(See Appendix A for an historical note on 
enlargers.) 


E the early days of photography, the 


Historically, cameras were manual 
devices, and the photographer was expected 
to load the film, set the lens aperture, set 
the shutter speed, adjust the lens to obtain 
the desired focus, and even cock the shut- 
ter before making the exposure. The larger 
professional cameras often had film hold- 
ers for sheet film, while smaller cameras 
that were available later used roll film that 
could be cranked to move fresh film into 
position. Also, professional cameras usu- 
ally had interchangeable lenses so that the 
photographer could vary the magnification. 
In all cases the photographer had to deter- 
mine in advance the exposure required to 
obtain a satisfactory image with the avail- 
able light and the speed or sensitivity of the 
film emulsion. 

The modern history of the camera 
consists of a series of steps toward total 
automation. Now in the 21st century most 
consumer cameras are automated; in fact, 
the simpler ones are called point-and-shoot 
cameras. This degree of automation exceeds 


FIGURE 3.1. A Rochester Camera Co. PREMO SR 5 in. x 7 in. portable camera (1896). (Camera 


from the collection of Denise Tolley.) 
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anything the pioneers could have imagined. 
We not only have automatic focus and 
exposure, but there is image stabilization 
to counteract camera motion during expo- 
sure, “face recognition” so that exposure 
and focus will be optimized for the areas of 
most interest in a scene, and much more. 

In spite of all the automation, knowledge 
about the way cameras work is still helpful 
and in some cases essential for high qual- 
ity photography. Consider also the problem 
of choosing a camera from the bewilder- 
ing array of cameras with different features 
that are available on the market. Why buy 
an expensive, large camera when a com- 
pact one fits in a shirt pocket and still offers 
high resolution? The answer depends on the 
needs of the user, but an informed decision 
requires some knowledge of the options 
and trade-offs. Of course, many people find 
that the cameras in smart cell phones are 
adequate for their purposes, and they opt 
not to purchase any other camera. 


3.3 The Lens 
he lens, necessary for forming an 
image on sensitized film or a sensor 
in a camera, is characterized by a 
focal length and an aperture. For simplicity, 


I only consider the simple, thin lens in this 
chapter. 


Definition: The focal length fof a thin lens 
is the distance behind the lens that rays 
from a distant object, i.e. parallel rays, are 
focused to form an image. 


For closer objects the image plane is 
located a greater distance from the lens 
than the focal length. See Figure 3.2. The 
focal length fof the lens determines the size 
of the image and, therefore, larger cameras 
require lenses with greater focal lengths. I 
will show in Chapter 4 that the “normal” 
lens for a camera has a focal length that is 
approximately equal to the diagonal dimen- 
sion of the film or sensor. This choice of 
focal length assures that prints viewed from 
a convenient distance will appear to have 
the natural field or angle of view. 

It is interesting to note that the theory 
of lens design was developed before the 
advent of photography. In fact, a few well- 
designed lenses were available before 1839. 
The earliest lens to be properly designed 
was the Wollaston Landscape lens of 1812, 
which was still being produced for low 
cost cameras in the mid-20th century. The 
improvement of camera lenses was very 


Lens 


k— | 


FIGURE 3.2. Parallel light rays from the left focus at the distance f behind the lens. 
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slow in the decades after 1840, however, 
because opticians usually worked by trial 
and error rather than the systematic appli- 
cation of principles. Lens design is actually 
a very difficult process that involves many 
compromises. Light rays must be bent and 
directed to exactly the correct point on 
the image, and rays of all colors should be 
focused at the same point. There are, in 
fact, six independent optical aberrations 
that need to be corrected to produce a high 
quality lens; and the reflections from the 
glass surfaces also need to be controlled. 
Optical aberration refers to deviations 
from the behavior of an ideal lens. This 
topic will be considered in detail in Chapter 
8. Fortunately, efficient lens design is now 
greatly aided by computer programs that 
can quickly trace light rays through any 
combination of lenses. Once an acceptable 
design is found it can be scaled up or down 
in size. Of course, the construction of high 
quality lenses is still a demanding task, and 
the required materials such as low disper- 
sion glass can be very expensive. (Here 
dispersion refers to the way color, actually 
wavelength of light, affects the focusing 
ability of glass.) 

A photographic lens is said to have a 
speed that determines how much light can 
be delivered to the image plane in a given 
time, i.e. the exposure. Lenses are equipped 
with variable apertures, but each lens has 
a maximum aperture. Here aperture refers 
to the diameter of the opening that admits 
light. Lens speeds are also described by stops 
or F-numbers. The adjustable F-number 
or N value of a lens may be listed as “f/N” 
where fis the focal length of the lens. This 
is actually an expression for the aperture as 
a fraction of the focal length. The minimum 
value of N, or the brightest stop, is usually 
engraved on the lens mount in the form 
1:N. 


Definition: The F-number N is the ratio 
of the focal length, f to the diameter of the 
aperture, 6. Thus, we have the formula N = 


fio. 


For example, if the focal length is f = 
20 mm and the maximum aperture is 6 = 
5 mm, N is 20 mm/5 mm = 4.0. Then one 
would say this is an f/4.0 lens or the lens bar- 
rel might be labeled 20 mm 1:4. Notice that 
the larger the aperture, the smaller the value 
of N. The F-number is actually a measure 
of the relative aperture so that a value of N 
ensures the same intensity of light on the 
sensor regardless of the focal length of the 
lens. In the special case of close-up photog- 
raphy, the speed or brightness of the lens is 
diminished because the image plane is far- 
ther from the lens than one focal length. An 
effective N can be defined in terms of the 
image magnification to take into account 
this effect (see Chapter 12). 

The amount of light admitted to the 
camera is proportional to the area of the 
aperture. Recalling that for a circle, Area = 
m(6/2)? we see that when the aperture 6 is 
decreased by a factor of 2, Nis doubled and 
the exposure is decreased by a factor of 4. 
Note that when the diameter, or aperture, 
is increased by a factor of V2 = 1.414 the 
area doubles. For this reason the common 
Ns listed on lenses, 2.0, 2.8, 4.0, 5.6, 8, 11, 
etc., correspond to changes by a factor of 
1.414, and changes of exposure by a fac- 
tor of 2. Each step is called a “stop” Lenses 
with small Ns, i.e. larger apertures, tend to 
be more difficult to make and are usually 
more expensive because of the measures 
that must be taken to limit aberrations. The 
larger apertures permit proper exposures 
to be made with higher shutter speeds 
(shorter shutter open times), but there are 
other trade-offs. In Chapter 11 I define the 
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depth-of-field and show how it is related to 
the lens aperture. 


3.4 Film and Sensors 


ineteenth-century photographers 
Nee had to sensitize their own 
photographic plates and work with 
hazardous chemicals. In the daguerreotype 
process, the image was made on silver- 
plated copper that was coated with silver 
iodide, and developing the image required 
warm mercury. By the time of the American 
Civil War the wet plate collodion process 
was popular. That method was much less 
expensive but required the photographer to 
coat a mixture of collodion, i.e. gun cotton 
dissolved in alcohol or ether, with appropri- 
ate light sensitive chemicals on a glass plate, 
to expose the plate while it was wet, and to 
develop the plate before it could dry. The 
rigors of photography during this period 
certainly limited its appeal to amateurs. 
Convenient amateur photography 
should be dated from 1895, when the 
Eastman Kodak Company first produced 
roll film with a paper backing so that it could 
be loaded in broad daylight. Since that time, 
numerous film sizes have come and gone, 
the dominant survivors being 120 and 135 
(the standard 35 mm).' These are arbitrary 
numbers that are not directly related to 
film size. Cameras that use 120 film make 
images that are 6 cm in height and range 
from 4.5 cm to 9 cm in width. There is also 
a panoramic camera that provides an image 
width of 17 cm. These cameras are all called 
“medium format” In contrast, 135 film is 
35 mm in width, but this includes sprocket 
holes. The actual image size is usually 
24 mm x 36 mm, and this is now called a 
full-frame (FF) 35 mm image. 
In film-based photography, both black 
and white and color, the image is captured 


in silver halide grains, and the sensitivity of 
the film is related to the size of the grains 
regardless of film size. This results from the 
fact that a few photons activate a grain so 
that it can be developed to make a silver 
spot on the negative. The larger grains con- 
tribute more to the darkness or density of 
a negative than the small grains do. Grain 
is, of course, more noticeable with 35 mm 
film than with larger films because greater 
enlargement is required to obtain a desired 
print size. High sensitivity to light requires 
large grains, but an image composed of 
large grains has lower resolution. By resolu- 
tion, I mean the ability to distinguish closely 
spaced objects in an image. Other things 
being equal, one may obtain higher qual- 
ity images by using a larger film format, but 
with the larger format comes a larger, heav- 
ier, and usually more expensive camera. 

The characteristic of film-based pho- 
tography is that grains of silver salts sample 
the intensity of the image projected by the 
lens, and the developed grains represent the 
image on film or paper. In comparing film 
photography with digital photography, it is 
important to note that the film grains are 
randomly placed. This leads to a sand-like 
texture in greatly enlarged photographs. 
When the texture is obvious, the image is 
said to be “grainy” Color film is much more 
complicated than black and white. The sil- 
ver grains must be replaced with color dyes, 
and three layers are required for the colors 
necessary to produce color slides and color 
negatives. Color film design and the chem- 
istry ofimage development are described in 
Appendix G. 

In the 21st century, digital photogra- 
phy has introduced a range of sensor sizes 
to replace photographic film. The sensors 
are microelectronic chips (CMOS or CCD, 
see Appendix H) that consist of a regular 
array of photo detectors that effectively 
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convert photons into electrons so that elec- 
trical charge can be accumulated in each 
picture element. This in turn permits the 
image to be stored in a computer memory. 
The area associated with each detector is 
defined as a pixel, which is short for “picture 
element” 


Definition: A pixel is the smallest element 
in an image, and can be thought of as a 
single point in an image. A pixel also rep- 
resents an area on the camera sensor that 
contains a light detector. 


Pixels are usually square in shape but 
may be rectangular or even octagonal, as in 
the super CCDs produced by Fujifilm. Since 
chips with large areas are very expensive to 
make, most consumer digital cameras thus 
far use very small sensors. Chapter 16 cov- 
ers sensors in detail. Table 3.1 shows a few 
typical sensors to illustrate the sizes. There 
are several variants of APS-C sensors. The 
one shown is for a Canon sensor. Mobile 
phone sensors are usually quite small. In 
recent phones, 1/3 in. is typical. The num- 
ber of pixels for a given sensor type may 
vary greatly. 

The Type column in Table 3.1 con- 
tains some arcane nomenclature that is a 


carryover from TV tube sizes in the 1950s. 
It is a diameter measurement not simply 
related to the size of the sensor, but it turns 
out to be about 50% larger than the diago- 
nal of the rectangular sensor. The first four 
sensors listed are found in recent compact 
point-and-shoot cameras. However, with 
so-called mirrorless camera designs we 
find sensor sizes up to FF in small camera 
bodies. Of course, the lens sizes and weights 
increase with sensor size. The 35 mm film 
size has carried over to digital sensors and 
is still known as full frame, or FF. Digital 
medium format dimensions vary among 
manufacturers but are always smaller than 
medium format film dimensions. 

The pixel width (or pixel pitch) in the 1/3 
in. sensor depends on the total number of 
pixels in the sensor. For 8.0 megapixels the 
pitch is about 1.48 um. The APS-C and full 
35 mm sensors are found in digital single 
lens reflex (DSLR) cameras, some mirror- 
less or small system cameras, and in a few 
special purpose fixed lens cameras. Here 
the pixel pitch is much larger. It amounts to 
8.42 um and 4.13 um for 12.2 megapixel and 
50.6 megapixel FF cameras, respectively. 
The difference in pixel areas is even more 
striking. The ratio of pixel areas for the 35 
mm, 12.2 megapixel sensor, to the 1/3 in., 


TABLE 3.1. 
Typical Sensors for Digital Cameras (2015) 


Type Aspect Ratio Width (mm) Height (mm) Diagonal (mm) 
1/3" 4:3 4.8 3.6 6.0 
125 4:3 5.76 4.29 7.18 
Wile” 4:3 76 5.7 9.6 
1.0” 4:3 13.2 8.8 15.9 
4/3" 4:3 17.3 13.0 21.6 
APS-C 32 222 14.8 27.04 
FF (35 mm) 3:2 36.0 24.0 43.27 
Medium Format 4:3 43.8 32.8 54.72 
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FIGURE 3.3. Sizes of some common sensors (mm). 


8.0 megapixel sensor, is approximately 32. 
At the same exposure (light level, F num- 
ber, and shutter time) a pixel in the 35 mm 
sensor will receive more than 32 times as 
many photons as a pixel in the 1/3 in. sen- 
sor. However, it should be noted that the 
photon detectors are smaller than the pixel 
area. Think of the pixel in a CMOS sensor as 
the square top of a deep box with the detec- 
tor occupying part of the bottom of the box. 
In order to increase the efficiency of light 
detection, a micro lens is used in each pixel 
to focus light onto the detector. This situa- 
tion has improved with some recent CMOS 
sensors that have backside illuminated 
designs that bring the light detectors to the 
front of the sensor. 

Each sensor type has advantages and dis- 
advantages that go far beyond a simple pixel 
count. I have already alluded to the differ- 
ence in signal strength resulting from the 
number of photons detected. The signal-to- 
noise (S/N) ratio is typically lower for a small 
sensor. In other words, pixels that should 
be recording the same intensity may show 
different signals because of fluctuations in 
the number of photons being detected. The 
photons arrive randomly, just as raindrops 
do, and the same number does not reach 
each collector in a given amount of time. 
This is analogous to catching raindrops in 


small buckets. The buckets (detectors) may 
receive more or less than the average num- 
ber of droplets (photons). Thus, when the 
average number of photons is 36, the actual 
numbers will range from about 30 to 42, 
and, in general, number fluctuation is about 
equal to the square root of the average num- 
ber, e.g. V36 = 6. The practical consequence 
of this is that the effective film speed (ISO 
number) must be kept low for small detec- 
tors. (For present purposes the ISO number 
is just a camera setting that determines how 
much the signal is amplified.) This means 
that larger exposures are required to make 
up for the small areas of the detectors and 
to collect comparable numbers of photons 
in the smaller pixels. As we shall see in 
Chapter 16, compact cameras produce good 
low-noise images at low sensitivity, e.g. ISO 
100, but above ISO 400 the noise (digital 
graininess) is often unacceptable. On the 
other hand, digital DSLRs that have larger 
pixels can produce adequate images at ISO 
12,000 or even higher. The ISO sensitivity 
setting basically determines the number of 
photons required to produce a certain sig- 
nal level. A precise definition of ISO is given 
in Chapter 16. 

In practice, neither the graininess of film 
nor the size of the pixels completely deter- 
mines resolution. In subsequent chapters, 
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resolution will be defined, and the fac- 
tors affecting resolution will be explained. 
In particular, the quality of lenses and the 
effect of diffraction will be considered. We 
will see that everything in photography 
involves compromises. In order to create an 
image, we face limitations imposed by the 
laws of physics, the availability of materi- 
als, and the present state of technology. The 
desired properties of the final image must 
guide our choices of equipment and set- 
tings. The appearance of an image depends 
on the perspective (position of the lens), the 
field of view, the depth-of-field, the duration 
of the exposure, and issues involving qual- 
ity. Each of these aspects must be treated 
before we can understand how to obtain 
equivalent images with different cameras. 


3.5 Digital Camera Design 


igital cameras in the 21st century 
are evolving quickly. In 2010 most 
camera sensors were found in cell 
phones, and cell phone cameras had attained 
such high quality that the market for small 


(a) 


Pentaprism 


Focusing glass 


Image sensor 


(point-and-shoot) digital cameras was deci- 
mated. The camera manufacturers reacted 
to the market shift by introducing greatly 
enhanced small cameras with 1 in. and even 
4/3 in. sensors that far outperformed cell 
phone cameras. Some enthusiast compact 
cameras also offered wide zoom ranges but 
at increased cost and weight. 

As will become clear in later sections of 
this book, image quality depends on sen- 
sor size for a number of reasons. Therefore, 
serious amateurs and professionals still 
require large cameras, and the popular sen- 
sor sizes range from four-thirds to medium 
format. The first digital cameras were essen- 
tially film cameras with sensors in place of 
film. The smaller cameras might include an 
LCD panel on the rear to display the image 
captured by the sensor and a small optical 
viewer on the top. The digital SLR (DSLR) 
cameras, of course, maintained the mirror 
and pentaprisms of their film counterparts. 
This design, illustrated in Figure 3.4(a), 
permits the user to look directly through 
the camera lens to compose and focus the 
scene. 


(b) 


Electronic 
viewfinder 


Image sensor 


FIGURE 3.4. Camera designs: (a) digital single lens reflex (DSLR) and (b) rangefinder style mir- 


rorless camera. 


THE CAMERA—AN INTRODUCTION 


At first, consumer-level DSLR cameras 
only had APS-C size sensors because of 
the high cost of the larger 35 mm (FF) and 
medium format sensors. With decreasing 
cost of sensors, FF digital cameras became 
popular. DSLR cameras continued to be 
supported by Canon, Nikon, and Pentax 
because of their extensive collections of 
compatible lenses and their highly evolved 
systems for exposure and focus tracking. 
DSLR cameras with 4/3 in. size sensors 
were introduced later in an effort to control 
size and weight of cameras and lenses. Also, 
medium format sensors were produced, 
although not quite as large as 6 cm x 6 cm 
and 6 cm x 7 cm film formats. 

The rangefinder type of focusing was 
used in some cameras in the film era, and 
Leica continued using rangefinders into 
their first decade of digital cameras. That 
design permits a lighter and less complicated 
camera since the mirror and pentaprism or 
SLR cameras are not required. Technology 
has now progressed to the point that Leica 
and a number of other manufacturers can 
offer mirrorless cameras with electronic 
viewfinders based on small, but high- 
resolution, LCD screens in addition to 
back panel LCDs as illustrated in Figure 
3.4(b). 

Inspection of Figure 3.4 shows that the 
DSLR design is much more complex than 
the mirrorless design. The mirror must 
swing up out of the way when a photo is 
taken, and a pentaprism is required to 
reflect and orient the image so that it can be 
presented in the viewfinder. Light from the 
mirror is used in the autofocus system, and a 
focusing glass screen is also required. In this 
design the LCD panel can only display an 
image after exposure, and a live-view mode 
requires that the mirror be raised. The space 
required for the mirror and pentaprism 
assembly introduces another problem. The 
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lens mount cannot be close to the film or 
sensor. Accordingly, all DSLR lenses have a 
retrofocus design that acts as if a lens ele- 
ment is close to the sensor. The resulting 
lenses are larger and more complicated. 

At the time of this writing, DSLR cameras 
are still competitive, especially for sports 
and nature photographers, because of their 
superior focus tracking, optical viewfinders, 
and repetition rates. However, because of 
size, weight, and simplicity, many photogra- 
phers find mirrorless cameras compelling. 
The electronic viewfinders found in high- 
end mirrorless cameras do not appeal to all 
photographers, but they permit the display 
of real-time histograms and other informa- 
tion. Also, focus based on the sensor avoids 
calibration of an autofocus system as well as 
providing automatic correction of focus for 
IR modified cameras. 

There is also a disruptive aspect to the 
mirrorless camera technology, since the 
thinner bodies permit lenses from DSLR 
manufacturers to be used with adaptors. As 
lens adaptors permit more and more camera 
features to be fully utilized with non-native 
lenses, even more photographers will be 
attracted to new mirrorless designs. As an 
indication of future trends, two companies 
are even offering medium format mirrorless 
cameras. 


Notes 


1. More than 30 Kodak film types have 
been discontinued. The most recent 
ones to be removed from production by 
Kodak were 127 (1% in. x 2% in.) and 620 
(2% in. x 34 in.) in 1995, and 828 (28 mm 
x 40 mm) in 1985. A few professionals 
and serious amateurs still use 4 in. x 5 in. 
sheet film. The most recent film intro- 
duction was number 240, Advanced 
Photo System (APS), in 1996. This film, 
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which provides a transparent magnetic halide film, did not succeeded in displac- 
recording medium for image informa- ing 35 mm as the most popular amateur 


tion in addition to the traditional silver film. 


CHAPTER 4 


Images: What Is Perspective? 


When, in early fifteenth-century Florence, 
Filippo Brunelleschi contrived the first painting in 
“true perspective,” he raised a sensation. 


—STEPHEN L. TALBOTT 


l like the fact that the word [perspective] 
literally means “your opinion.” Where you 
place the camera is both an assertion of 
your camera’s view point and your opinion. 


—MAHESH VENKITACHALAM 


irst let’s agree that by image we mean 
E two-dimensional representation 

of the view defined by the camera 
lens and the sensor as shown in Figure 4.1. 
According to Rudolf Kingslake, a photo- 
graph is “a two-dimensional projection of 
a three-dimensional object, the projection 
lines all passing through a common center” 
This brings up the concept of perspective, 
by which we mean the way things appear 
to the eye based on their positions and dis- 
tances. Basically, we want to represent a 
scene in two dimensions so that it gives the 
same impression of sizes and distances that 
is experienced in nature. 

Some photographs have a natural appear- 
ance, while others seem distorted because 
nearby objects appear too large or distant 
objects are flattened. These are natural 


effects having to do with the perspective 
point of the image and the tilt of the camera. 
Other distortions in which straight lines 
appear to be curved are different and will be 
discussed later. The first thing to recognize 
is that, for a lens in a particular position, 
the perspective of the resulting image is 
independent of the focal length of the lens. 
However, for distant objects, the size of the 
image will be approximately proportional to 
the focal length. The only way to change the 
perspective is to move the camera. This effect 
is illustrated in Figure 4.2, where the camera 
was not moved, but the lens was changed 
to obtain focal lengths from 10 mm to 135 
mm. Recall that the focal length fis the dis- 
tance behind a lens at which parallel rays are 
focused. The inverse (1/f) is defined as the 
power of the lens in units of diopters when 
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FIGURE 4.1. The perspective point is at P, and it may be the eye of the observer or the lens of 
a camera. 


Toe 


FIGURE 4.2. The effect of changing the focal length while maintaining the perspective. The cam- 
era was a Canon XTi (22.2 mm x 14.8 mm sensor). 
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fis measured in meters. This definition will 
be generalized for thick lenses in Chapter 
9. (These comments refer to cameras where 
the lens plane is parallel to the surface of 
the detector. Shift and tilt lenses are special 
cases and require additional analysis.) 

The photographs shown here were taken 
with rectilinear lenses corrected to repro- 
duce straight lines as straight lines in the 
image. They approximate linear perspective 
where horizontal lines perpendicular to the 
line of sight remain horizontal, but other 
sets of parallel lines converge to a vanishing 
point at infinity. 

A normal or natural perspective depends 
on the angle of view of a photograph rela- 
tive to that of the original scene. According 
to Kingslake’s analysis, the region of sharp 
vision of the human eye has an angular width 
of about 20° or a half-angle of 10° and motion 
is detected over a range of about 50°. The 
secret to obtaining a normal view is to view 
a photograph from its perspective point. In 
that way the eye has the same relationship to 
objects in the photograph that the lens did to 
objects in the original scene. The perspective 
point of a negative or an image sensor is the 
center of the camera lens or a distance that 
is approximately the focal length of the lens. 
For an enlarged photograph, the perspective 
distance is the lens distance multiplied by the 
degree of enlargement. 

Suppose, for example, that a 35 mm 
camera with a 50 mm (#2 in.) focal length 
lens produces a negative (1 in. x 1.5 in.) that 
is used to make an 8 in. x 10 in. print. The 
enlargement factor is approximately 8 in./1 
in. = 8, and the proper distance to view the 
print is 2 in. x 8 = 16 in. This is a normal 
viewing situation, but consider a wide-angle 
lens with a focal length of 17 mm (0.67 in.) 
that is used with the same camera body. The 
proper viewing distance for an 8 in. x 10 in. 
print becomes 0.67 in. x 8.0 in. = 5.36 in. 


Gy = 
Unfortunately, most 8 in. x 10 in. prints are 
viewed from approximately the same dis- 
tance regardless of the point of perspective, 
and thus the apparent perspective suffers 
and one perceives distortion. 

So what is a “normal lens”? A normal lens 
is one that produces photographs that, when 
viewed normally, will put the eye close to the 
proper perspective point. A lens produces a 
circle of illumination and a circle of good 
definition. Since negatives and sensors are 
square or rectangular in shape, the diagonal 
must fit inside the circle of good definition. 
By definition, a “normal lens” has a focal 
length approximately equal to the diagonal 
dimension of the sensor. This choice ensures 
that the camera’s angle of view will be simi- 
lar to the angle of view of the human eye. 
For a 35 mm camera, the normal lens has a 
focal length of about 43 mm, and a simple 
computation shows that the angular (diago- 
nal) field of view has a half-angle of 26.5° 
[O = tan”*(0.5)] or a full angle of 53°. This is 
somewhat larger than the eye can view at a 
glance, but it is an acceptable compromise, 
since prints are usually viewed from beyond 
the perspective point 

Wide-angle lenses, of course, have 
shorter focal lengths than the normal 
lens. A typical wide-field lens has a diago- 
nal angle of view of about 75° or a focal 
length of 28 mm (FF 35 mm equivalent). 
Ultra wide-angle lenses have fields of view 
from about 90° to 100° and focal lengths of 
about 17 mm to 20 mm (FF equivalent). 
These lenses are often blamed for strange- 
looking, apparently distorted, photographs. 
The problem is that the proper perspec- 
tive point for viewing wide-angle prints 
is inconveniently close to the print. This 
effect can be alleviated by making larger 
prints or murals. Figure 4.3 illustrates the 
change in perspective when the same set of 
stairs was photographed at close range with 
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FIGURE 4.3. A set of stairs photographed with a 10 mm lens (top) and a 190 mm lens (bottom). 
The camera was a Canon XTi (22.2 mm x 14.8 mm APS-C sensor). 


a wide-angle lens and from a considerable 
distance with a telephoto lens. 

With digital cameras and computer- 
assisted image processing it is easy to stitch 
images together to obtain extremely large 
fields of view. Each image in the resulting 
panorama usually has a perspective point, 
and the way to avoid the appearance of an 
unnatural image is to print or project a very 
large picture. 

There are also narrow field lenses known 
as “telephoto” lenses. Any focal length 
longer than the “normal” focal length will 


give a telephoto effect, but true telephoto 
lenses have a compact design in which a 
negative lens element is inserted to enlarge 
the image. By definition, the length of a true 
telephoto lens from the front element to the 
film plane is much less than the focal length. 

Of course, most cameras for amateurs 
are sold with zoom lenses. These lenses 
smoothly change focal length with a twist, 
a push/pull motion, or the pushing of a but- 
ton. This will not change the perspective of 
the image, but it will certainly change the 
perspective point from which photographs 
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should be viewed. Fixed focal length lenses 
are known as prime lenses. 

Our final point about perspective 
involves camera tilt. It is common knowl- 
edge that, when a camera is tilted up, the 
image will be distorted so that vertical lines 
tend to converge, buildings appear to lean 
backward, etc. This effect is often consid- 
ered undesirable, and methods are sought 
to avoid or to correct for it. Holding the 
camera perfectly level will remove the dis- 
tortion, but may clip the tops of objects and 
may miss points of interest. Professional- 
view cameras avoid the tilt problem by 
raising the lens while holding the lens plane 
parallel to the film plane, and there are spe- 
cial purpose tilt/shift lenses available for 
interchangeable lens cameras from several 
manufacturers. Lens shift provides correc- 
tion for camera tilt but does not change the 
perspective. The same effect can be achieved 
by cropping a wide-angle image taken with 
a camera that is held level. 

Corrections for tilt in existing negatives 
can sometimes be made by tilting the paper 
holder in a photographic enlarger. Enlargers 
are less common in the digital age, but we 
have much more powerful tools. Image 
processing software such as Photoshop 
can transform (distort) an image to cor- 
rect for the convergence of lines. Computer 
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programs can also correct for lens aberra- 
tions and distortions and can even convert 
images taken with fisheye lenses to rectilin- 
ear form. 

Strangely enough, extreme camera tilt is 
often considered pleasing. Also, sideways 
tilt that makes horizontal lines converge at 
the horizon is not considered a defect. This 
effect is so common and accepted that art- 
ists often use geometric projections to build 
this kind of perspective into their images. 
In this and all the other cases mentioned, 
a natural view is obtained by using one eye 
at the perspective point. The use of two 
eyes (binocular vision) gives an unnatu- 
ral appearance for small images. We don't 
notice this problem with natural scenes, and 
we can avoid it by making large images. 

In closing this discussion, I note that 
our perspective point is determined when 
we select a seat in a movie theater, but that 
position is seldom the perspective point of 
the movie photographer. 


Further Reading 


Kingslake, R. (1951). Lenses in Photography. 
Rochester, NY: Case-Hoyt Corp., chap. 1. 
Solso, R. L. (1994). Cognition and the Visual 
Arts. Cambridge, MA: MIT Press, chap. 8. 
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CHAPTER 5 


Why Does a Camera Need a Lens? 


Look and think before opening the shutter. 
The heart and mind are the true lens of the camera. 


—YOUSUF KARSH 


think the best way to discover the need 
IE lenses is to analyze a famous lens- 
less camera, the pinhole camera. This 
is nothing more than a small version of the 
camera obscura that played such a large role 


in the history ofart and photography. All we 


need is a light-tight box with a pinhole in the 
center of one wall and a photographic plate 
attached to the opposite wall. As illustrated 
in Figure 5.1, the formation of an image can 
be explained with nothing more than ray 
optics. That is, we assume that light beams 


FIGURE 5.1. Illustration of a pinhole camera aimed at the Temple of Wingless Victory (Athens). 
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move in a straight line, and we perform ray 
tracing. A ray of light from any point on 
an object passes through the pinhole and 
strikes the plate on the opposite wall. 

It is obvious from the figure that a point 
on the object will become a small circle in 
the image because the pinhole aperture must 
have some diameter to admit the light. It is 
a catch-22 situation. If we make the pinhole 
smaller, the light reaching the image will be 
diminished; and the exposure will take a 
longer time. On the other hand, if we enlarge 
the pinhole, the resolution will suffer. At first 
it appears that there is no optimum size, and 
we just have a trade-off between resolution 
and the time required to expose the plate. 
But that is not true. If we try smaller and 
smaller pinholes, at some point we find that 
the resolution actually decreases. The rea- 
son for this inconvenient development is the 
ubiquitous diffraction phenomenon. 

In the limit of very small apertures, the 
image of a distant point, say a star, becomes 
a circular disk with faint rings around it. 
This bright central region, called the Airy 
disk, is illustrated on the right side of Figure 
5.2. The diameter of the disk measured to 
the first dark ring is equal to 2.441N, where 
A is the wavelength of the light and N is the 
ratio of focal length to aperture, f/6, defined 


aperture 


= 


in Chapter 4 as the F-number. The spot size 
for a distant point without diffraction is just 
equal to the aperture diameter ô. Therefore, 
the spot size will increase for both large and 
small pinhole sizes. 

The optimum pinhole size will occur 
when the contributions of these two effects 
are equal. Therefore, 

6 = 2.44 1f/6 (5.1) 
and we conclude that the optimum aperture 
size is equal to: 


ô, = V244 Af 


opi 


(5.2) 


Since A = 500 mm in the middle of the 
visible spectrum, we find that 


ô „= 0.035 Vf (mm) (5.3) 


op 


for ô „and f measured in mm. When f= 100 
mm, the optimum pinhole has the diameter 
Ô pi = 0.35 mm. This is approximately the size 
ofa number 10 needle. A formula attributed 
to Lord Rayleigh improves this calcula- 
tion by the substitution of 3.61 for 2.44 in 
Equation (5.1). 

This derivation is only approximate, and 


it fails badly for nearby objects. It is clear 


sensor 


FIGURE 5.2. The effect of a small aperture on light incident from the left. The vertical scale is 
greatly exaggerated to reveal the intensity distribution in the diffraction spot. 
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that light rays from a bright point near the 
pinhole will diverge through the pinhole 
to make a spot on the film that is larger 
than the pinhole. This results in a loss of 
resolution for the nearby objects, and 
the realization that the optimum size for 
the pinhole depends on the distance from the 
pinhole to the object being photographed. 

Photography with the pinhole camera 
requires an exposure time that depends on 
the F-number (N) of the aperture. From the 
definition we find that N = 100 mm/0.35 
mm or //287 for this camera with its opti- 
mum pinhole size. How does this compare 
with a typical lens based camera? A typical 
aperture setting for outdoor photography is 
J/8, and we have found that every time the N 
is doubled, the intensity is cut by a factor of 
four. f/8 must be doubled five times to equal 
J/287, and this indicates about 4° = 1,024 
times as much light intensity on the image 
plane as with the pinhole camera. The rec- 
ommended exposure time for typical (ISO 
100) film in hazy sun light is 1/125 sec at 
//3. The corresponding exposure with the 
optimum pinhole camera would be about 
8 seconds. 

Thus, the pinhole camera at best gives 
poorly resolved images and requires very 
long exposure times. It does give interest- 
ing effects since any focal length is possible, 
and the images show no aberrations or dis- 
tortion. The long exposures also can offer 
unusual effects. For example, during long 
exposures objects can move through the 
field of view and leave no trace in the image. 
In fact, there is a lot of interest in pinhole 


cameras by hobbyists. However, pinhole 
cameras are toys compared to cameras with 
lenses. 

One other point is worth noting. The 
ray tracing approach described above 
assumes that rays are undeviated (the path 
is unaffected) by the pinhole other than the 
spreading effect of diffraction. This works if 
the same medium, such as air, fills the cam- 
era and the outside space. Suppose instead 
that the space between the pinhole and the 
film is filled with water. A ray passing from 
air to water is subject to refraction and the 
path is bent toward the optical axis (a line 
through the pinhole and normal to the film 
plane). The unexpected result is a fisheye 
view of the world. This effect is discussed 
and illustrated in Chapter 10. An air-filled 
pinhole camera with water outside would 
be the reverse situation, and should give 
a telephoto effect. As far as I know, this 
arrangement has not been demonstrated. 

Here is what lenses do: They permit sharp, 
high-resolution photos to be made with 
large apertures. The large apertures mean 
that adequate exposures can be obtained 
with high shutter speeds (short exposures). 
In addition, the effects of diffraction can 
be reduced and the region of sharp focus 
(depth-of-field) can be controlled. Thus, we 
see that lenses have transformed a curiosity 
into an image-making miracle. 


Further Reading 


Grepstad, J., Pinhole Photography: http:// 
jongrepstad.com/pinhole_photography/ 
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CHAPTER 6 


Elementary Optics: How Do Lenses Work? 


Now, let’s have some fun. Let’s “fool the light,” 
so that all the paths take exactly the 
same amount of time. 


—RICHARD P. FEYNMAN 


and show how they work in simple 

lenses. Of course, science is not just a 
list of independent rules, and it should be 
recognized that these rules can be derived 
from more general laws of physics. The rules 
are not only consistent with Maxwell’s elec- 
tromagnetic theory, but they can be derived 
from the quantum theory of photons and 
electrons. Showing how all that works is 
really the fun part for those who enjoy 
physics and mathematics. An introduction 
to these ideas can be found in Appendix B. 

The most important fact is that light 
travels at the speed c = 2.997 x 10° m/s in 
a vacuum such as outer space, but that the 
speed appears to be reduced in transparent 
matter, such as water or glass.’ Each mate- 
rial has a characteristic index of refraction 
n that is greater than one, and the apparent 
speed of light in the material is equal to c/n. 
The refractive index depends on the fre- 
quency of the light and conditions such as 


È start with a few basic rules of optics 


the temperature and pressure that affect the 
properties of the material. Refractive indi- 
ces have been measured for all the materials 
of interest in optics and can be found in 
tables. For water at 25°C the refractive index 
is 1.33, and, therefore, the apparent speed 
of light in water is reduced to c = 2.253 x 
10% m/s. 

Here are the rules we need for the optics 
of light beams (rays): 


1. Ina material with a uniform index of 
refraction, light goes from place to 
place in a straight line. 


2. When light is reflected from a surface, 
the angle of incidence, denoted by 
0, is equal to the angle of reflection, 
denoted by 0. See Figure 6.1. 


3. When light passes from a medium with 
refractive index n, into a medium with 
refractive index n, the angles of inci- 
dence, 0,, and refraction, 0, are related 
by a formula known as Snell's law: 


32 oy CHAPTER 6 


n, sin 6, =n, sin 6, (6.1) 
where sin refers to the sine function of 
trigonometry. The geometry of refrac- 
tion is illustrated in Figure 6.2. 


4, Diffraction effects must be considered 
when light interacts with an aperture 
with a diameter that approaches the 
wavelength of the light. (This is not 
really a rule, but rather a warning that 
ray optics do not tell the whole story.) 


The rule that light travels in a straight 
line is essentially proved correct by the 


Source 


pinhole camera when the effects of dif- 
fraction are neglected. I will assume that 
it is correct and will use it in considering 
the other rules. The second rule is known 
as Hero’s law of reflection because he was 
the first person to publish it. This is Hero 
(or Heron) of Alexandria, who was a math- 
ematician in the first century CE. He is 
known for inventing the steam engine, 
but he was also interested in optics and he 
studied mirrors, probably polished metal 
or the surface of water. Hero’s Law can be 
derived from Fermat’s principle of least 
time. 


Detector 


Detector 


FIGURE 6.2. Illustration of Rule 3, Snell’s law of refraction. 
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That is to say, light of a given frequency 
moves between two points along the path 
that takes the least time. Pierre de Fermat 
was one of the best mathematicians in the 
world in the mid-16th century (CE). He 
is best known now for his “last theorem,’ 
which was proved only in the 1990s. 

The principle of least time is applicable 
to many problems in optics and deserves 
our attention. First, note that in a uniform 
medium, where the index of refraction is 
constant, the path that requires the least 
time is also the shortest path. If the path 
traverses regions where the refractive indi- 
ces are different, then the path requiring the 
least time must be chosen. This also leads to 
the idea that the optical path length is equal 
the physical path length multiplied by the 
appropriate index of refraction. 

In the reflection problem shown in Figure 
6.1, the path from A to C, i.e. path ABC, has 
its minimum length when x = L/2. From the 
Pythagorean Theorem we find that the path 
length ABC can be expressed as: 


P(ABC) = vh? + x? + vh? + (L - x)? (6.2) 

It is easy to verify that the minimum 
value of P(ABC) occurs when light reflects 
from the mirror at the midpoint between 
A and C, and that the value increases when 
any value of x other than L/2 is used. So the 
trial-and-error method works here, but the 
straightforward way to obtain the minimum 
path is to use the calculus.? 

Rule 3, Snell's law, is the important one 
for lens design. Willebrord Snellius was 
Professor of Mathematics at the University 
of Leiden when he discovered the law of 
refraction in 1621. For some reason he 
never published the law, and it only became 
known in 1703, when Christiaan Huygens 
revealed the equation in his Dioptrica. 
Snell’s equation can also be derived by 


Fermat's principle of least time. In Figure 
6.2 the light beam passes from medium 1 
(air) to medium 2 (glass). I will assume that 
n, < n, Which is appropriate for air (n,=1) 
and crown glass (n, = 1.55). The illustration 
shows that the path length in air is greater 
than the path length in glass, and this is 
necessary to minimize the time since the 
speed in air (c/n,) is greater than the speed 
in crown glass (c/n,). This is analogous to 
the situation of a life guard who must rescue 
a swimmer who is down the beach and out 
to sea. Since the guard can run faster than 
he can swim, he should not head directly 
for the swimmer but should run at an angle 
so that the distance covered on the beach is 
greater than the distance in the water and 
the total time can be minimized. 

The amount of time required for a light 
ray to go the distance L in a medium hav- 
ing the refractive index n is t = L/(c/n) = 
nL/c, and we call nL the optical path length 
as opposed to the physical path length L. 
Therefore, the time required for the light 
ray in Figure 6.2 to cover the path ABD is 
given by: 


t = [n vV + x + n, vh? + (L-x)]/c (6.3) 


The minimum value of t can be obtained 
by calculus or by a trial-and-error variation 
of x. It turns out that the minimum value of 
tis obtained when: 


nx 


vi? + x? 


n (L-x) 
Vi? + (L-a) 


(6.4) 


Equation (6.4) is identical to Snell’s law 
as described in Equation (6.1) since sin 6, = 
x/vh? + x° and sin 0, = (L—x)/ vh} + (L-«)?. 

Now consider the simple lens shown in 
Figure 6.3. Light rays emitted from a distant 
source on the left are focused on the image 
plane at a common point. According to 
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Lens 
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FIGURE 6.3. Light rays and a simple lens. Since the speed of light is lower in the glass lens than 
in air, all the paths take the same time to reach the focal point. 


Fermat’s principle of least time this happens 
because all of the rays go from the source to 
the focal point in the minimum time. This is 
possible because the apparent speed of light is 
lower in the lens, i.e c/n is less than c. The ray 
through the center of the lens has the shortest 
path in air but is delayed by passage through 
the thickest part of the lens and so on, and 
the lens is designed so that the optical path 
lengths and times for all of the rays shown 
are identical. From the starting points on the 
left-hand side other paths to the desired focal 
point would require more time. 

The principle of minimum time is a 
unifying principle that justifies, and can 
replace, the simple rules for reflection and 
refraction. But, does this principle always 
apply? The answer is yes, if the rule is prop- 
erly stated. The waves at extrema, requiring 
either minimum or maximum time and 
known as stationary paths, contribute the 
most to reflection. Ellipsoidal mirrors pro- 
vide examples of reflection from points 
requiring both minimum and maximum 
time. At extrema, a number of neighboring 
paths require almost the identical time and, 
therefore, constructively interfere. This idea 
is illustrated in Appendix B. The impor- 
tant idea behind all of optics is that a light 


ray is composed of photons but acts like a 
wave. In the classical picture, described by 
Maxwell’s equations, the wave represents 
the amplitude of the oscillating electric 
field associated with the light ray, and the 
square of the amplitude is proportional to 
the intensity of the light. In the quantum 
picture (QED) the light ray is described by 
a wavefunction, and the square of the abso- 
lute magnitude of this function gives the 
probability of finding a photon.’ This is all 
consistent because the intensity is deter- 
mined by the number of photons. 

So, fine, everyone agrees that we have 
waves, but how does this idea generate the 
rules of optics? The answer is the superposi- 
tion principle, namely the amplitude of the 
light wave is the sum of all light waves at the 
same time and position. The waves associ- 
ated with all the relevant light rays must be 
added to obtain the resultant wave before 
the amplitude is squared! It is only the set 
of rays that take the least time (or possibly 
the maximum time) that are in phase and 
contribute significantly to the intensity. This 
simple idea, that is explored in Appendix B, 
is the topic of Richard Feynman’s beautiful 
little book QED, The Strange Theory of Light 
and Matter. 
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Notes 


1. According to the theory of relativity, the 


speed of light is a constant in the uni- 
verse, so how can light have a reduced 
speed in matter, e.g. water or glass? In 
fact, it turns out that the index of refrac- 
tion can be less than one and even less 
than zero. So the refractive index is 
much more subtle than one might think. 
Feynman's interpretation of the index is 
as follows. The speed of light always has 
the value c. However, when light exits 
matter, the electric field that is detected 
results from all the electrical charges 
present. When light passes through 
matter, the electrons in all the atoms 
are disturbed and in turn contribute to 
the radiation field. The field we measure 
after the interaction of light with matter 
is the sum of all fields, and the resulting 


superposition exhibits a phase (location 
of nodes) that is approximately the same 
as that calculated by assuming that the 
speed of light in matter is given by c/n. 
The only thing that matters in optics is 
the phase of the wave, not how long it 
took to pass through the glass. 


. In the terminology of the calculus, the 


derivative of P(ABC) with respect to x is 
equal to zero at the minimum, and this 
determines the corresponding value of x. 


. Quantum electrodynamics (QED) is 


very well developed, and it permits cal- 
culations to be made with precision. The 
mathematics can be daunting, but the 
Nobel Prize-winning physicist Richard P. 
Feynman has explained how easy it is to 
see what the calculations mean. We don't 
have to understand why; we just need to 
accept the way nature works. The proce- 
dure is ridiculously simple (in principle). 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 


CHAPTER 7 


The Simple Thin Lens and What it Does 


A thin lens is a design tool, used to simulate a 
real lens during the preliminary stages of optical 


system design and analysis. 


—BRUCE WALKER 


7.1 Introduction 


he earliest lenses were made of pol- 
Th crystals, and it was not until 

the Middle Ages that glass lenses 
were produced. By the 13th century, glass 
lenses were good enough to be used in 
spectacles for the correction of presbyopia. 
More complicated optical instruments such 
as telescopes require lenses of sufficient 
uniformity that they can be used in pairs 
to obtain magnification. The master spec- 
tacle maker Hans Lipperhey of Middelburg 
improved lens grinding techniques and 
in 1608 was able to present a telescope 
to Prince Maurits in The Hague.’ Others 
claimed priority in inventing the telescope, 
but the first record we have of an actual 
instrument appears in Lipperhey’s patent 
application. The time was right, ideas were 
in the air, and someone was going to pro- 
duce good lenses and construct a telescope 
in the early 17th century. Unfortunately, 


Lipperhey didn’t get his patent; but he was 
well paid for his services, and he now has a 
lunar crater and a minor planet named for 
him. 

Everything changed in 1610 when Galileo 
made a telescope for himself and turned it 
on the heavens. He quickly discovered the 
moons of Jupiter and published his results. 
That same year Kepler got his hands on a 
telescope made by Galileo and worked out 
a new theory of vision based on two ideas: 


1. Light rays travel in all directions from 
every point in an object (Maurolico, 
16th century). 


2. A cone of light rays enters the pupil of 
the eye and is focused onto the retina 
(Kepler, 1604).? 


Prior to this time the operation of the 
eye was completely misunderstood. The 
revolution of 1610 opened a new world 
for scientific observations of objects both 
near (by the microscope) and far (by the 
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telescope). The history of optics, the science 
of vision, is beautifully described by Vasco 
Ronchi. (See recommendations for Further 
Reading at the end of the chapter.) 

The goal of this chapter is to show how 
points in subject/object space (in front of 
a lens) are related to points in the image 
space (behind a lens) for ideal lenses. With 
modern equipment it is relatively easy to 
grind and polish the surface of a circular 
glass plate into a spherical shape; that is 
to say, having the shape of a portion of the 
surface of a sphere. In a plane that contains 
the optical axis (axis of symmetry), a spheri- 
cal surface appears as a segment of a circle. 
To this day most camera lens elements are 
spherical in shape. I show in Chapter 8 that 
aspheric surfaces are sometime desirable, 
but they are much more difficult to fabri- 
cate. The next step is to show how a lens can 
be shaped so that it will have a certain focal 
length. This is, in fact, the starting point for 
understanding the design of all lenses even 
compound, multi-element types. 


7.2 Optical Surfaces 


o matter how complicated a lens 
is, it still consists of optical sur- 
faces separating regions that have 


different indices of refraction; and, in prin- 
ciple, all we need is Snell’s law of refraction 
to be able to trace the path of a ray of light 
through a lens. This idea is illustrated in 
Figure 7.1, where a ray of light (red) is inci- 
dent on a spherical surface at a distance h 
from the optical axis. The surface is char- 
acterized by the radius of curvature R; and, 
since the center of curvature is on the right 
side, the surface is convex and we assign the 
radius a positive value. This sign convention 
must be applied consistently. If the center 
were on the left side, the surface would be 
concave to the incident ray; and the radius 
would be negative. The effect of the sur- 
face is to change the direction of a ray if the 
refractive indices on the two sides (here n, 
and n,) are different. The calculation of the 
change in angle from 6, to 0, relative to the 
surface normal just requires the application 
of Snell's law. 

Snell's law (Chapter 6) tells us that n, 
sin 6, = n, sin 0, and it is very easy to com- 
pute the angle of refraction for a single ray 
passing through a surface. The calculation 
must, of course, be repeated for each sur- 
face of the simple lens and many surfaces 
in a compound lens. Furthermore, in order 
to characterize a lens, a representative set 
(cone) of rays from points on the object side 


FIGURE 7.1. A spherical surface with radius R. The dotted line from the center of curvature is 


normal to the surface. 
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must be used to sample the entrance surface 
of the lens. This is a tedious task, and nowa- 
days it is assigned to a computer. Such a set 
of calculations can predict the performance 
of a particular lens design, but does not give 
much guidance about how to design a lens. 
For that we need simple equations and rules 
of thumb to get started. Here, as in many areas 
of science, simplifying assumptions can lead 
to very useful approximate equations. For 
example, suppose that the rays remain close 
to the optical axis and that all ofthe angles are 
small. This means small enough that the sin 
0 and tan @ functions can be replaced with 0 
itself, where the angles are measured in radi- 
ans (211 radians equals 360°). This is called the 
paraxial approximation with which Snell's 
law becomes 1,0, = 1,0.. 

With the paraxial approximation, it is 
easy to derive equations for a simple thin 
lens. The optical power of the surface shown 
in Figure 7.1 is defined as P = (n, — n,)/R, 
and our first goal is to relate this power to 
the object and image distances p and q, 
respectively. Consider the situation illus- 
trated in Figure 7.2. Notice that the radius 
for the first (entrance) surface R, is greater 
than zero as in Figure 7.1 because the center 


Surface 1, power 
nn, 


20 R,>0 


of curvature is on the extreme right-hand 
side, and the radius of the second (exit) sur- 
face R, is less than zero because its center 
of curvature is far on the left-hand side. The 
first step is to apply Snell's law to derive an 
equation for the focusing power of surface 
1. We inspect Figure 7.1 and find that 0, = 
a, + y and 0, = y — a, Then the paraxial 
approximation permits us to use the follow- 
ing expressions for the angles: a, = h/p,, a, 
=h/q,,and y = h/R,. When these quantities 
are substituted into Snell’s law, we obtain 
the following equation for the power of 
surface 1: 


(7.1) 


Equations that describe the thin lens 
require that the powers of the two surfaces 
be combined. These derivations, which can 
be found in Appendix C, yield: 


(“-1)(=-=) 1,1 
Y R R p q 


If the space before the lens is occupied 
by air, so that n, is close to one, the focusing 


(7.2) 


Surface 2, power 


n.-n 
ee ai 


FIGURE 7.2. A simple convex-convex spherical lens. 
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power depends only the refractive index 
of the lens material and the radii of curva- 
ture. If the entrance surface of the lens is in 
contact with water, which has a refractive 
index of approximately 1.3, then the effec- 
tive power will be reduced. It can, of course, 
be compensated by decreasing the radius of 
curvature. Actually, I have assumed that the 
refractive indices in the object and image 
spaces are equal, and, in general, they are 
set equal to 1. 

Now suppose that the object is moved 
very far to the left so that 1/p approaches 
zero. In this limit q becomes equal to the 
focal length f and Equation (7.2) becomes 
the well-known Lens Maker's Equation: 


DGA 


1 


(7.3) 


In Equation (7.3) lens makers have a use- 
ful recipe for relating the properties of glass 
and the shapes of lens surfaces to the focal 
lengths they require. Another useful 
equation, (7.4), is obtained by noting that 
the left-hand side of Equation (7.2) is equal 
to the right-hand side of (7.3). Thus: 


a (7.4) 
f p q 

This thin lens equation is also called the 
conjugate equation, since it relates the con- 
jugate distances p and q to the focal length f. 
Equation (7.3) is wonderful for lens design- 
ers, but is usually not of much interest to 
photographers. On the other hand, the 
conjugate equation is very useful for under- 
standing the way lenses work. Right away 
we see that a simple lens can be focused by 
adjusting the lens to sensor distance q to 
accommodate the object to lens distance p. 
Furthermore, the magnification (the size of 
the image of an object relative to the actual 


size of the object) is equal to the ratio q/p 
and is, therefore, determined by the focal 
length. When the object distance p is much 
larger than the focal length f the magnifica- 
tion is approximately equal to f/p. Note that 
when p = q, the magnification m is equal to 
one and that both p and q are equal to 2f. 
These topics are explored in greater detail 
in Chapter 12. 


Caveat: In the paraxial limit the thin lens is 
ideal in the sense that a cone or rays from a 
point in an object will be focused into a cir- 
cular cone of rays that converges to a point 
(or to a diffraction limited disk). As the 
diameter of the lens (or the limiting aper- 
ture) increases so that the paraxial limit no 
longer applies, focused bundles of rays devi- 
ate from circular cones and take on a variety 
of patterns. The patterns are characteristic 
of spherical lenses and are associated with 
well-known aberrations. Chapter 8 covers 
aberrations and some ways to minimize 
their effects. 


7.3 Compound Lenses 


T he focusing power of the lens, taking 
into account both surfaces, is char- 
acterized by the quantity 1/f. Now 
suppose we have two thin lenses in contact 
as in Figure 7.3. What is the power of the 
combination? This is an easy question that 
can be answered by combining the conju- 
gate equations for the two lenses. First we 
introduce the labels 1 and 2 to obtain: 


1 & A 
and 
1 Lt. | 
E acei 
Pn h fh (7.5) 
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Object 


Image from 
lens 1 and 2 
> - combined 


- Image from 
lens 1 


FIGURE 7.3. A pair of thin lenses with exaggerated thicknesses. 


The image for the first lens falls to the right 
of the second lens and becomes its subject. 
Therefore, we can substitute —q, for p, in 
the second equation, where the minus sign 
indicates that the subject is behind (to the 
right) the lens. The equations can then be 
added to obtain: 


(7.6) 


The subscripts on the left-hand side can be 
dropped because the combined set of thin 
lenses only has one subject (object) distance 
and one image distance. The conclusion is 
that the focusing power of a set of lenses 
can be computed by adding their separate 
powers. From Chapter 6 we know that the 
quantity 1/fis defined as the power in diop- 
ters when fis measured in meters, and the 
rule is to add diopters when lenses are used 
in combination. A more involved deriva- 
tion shows that, when the two lenses are 
separated by the distance d, the equation 
becomes: 


7.4 Conclusion 


he simple thin lens provides a useful 

| model for understanding all lenses. 
The conjugate equation is the start- 

ing point for estimating the behavior of 
lenses, and it even applies to compound 


lenses under conditions that are discussed 
in Chapter 9. 


Further Reading 


Bi-Convex lenses, Interactive JAVA Tutorial: 
http://micro.magnet.fsu.edu/primer/ 
java/lens/bi-convex.html 

Fowles, G. R. (1989) Introduction to Modern 
Optics, 2nd Ed. New York: Dover Publ. 

Ronchi, V. (1991). Optics: The Science of 
Vision. New York: Dover Publ. 


Notes 


1. The development of lenses and even 
telescopes is credited to craftsmen, not 
scientists. In fact, philosophers dis- 
missed lenses, since their operation was 
not consistent with existing (European) 
theories of vision that were based on 
the emission of rays from the eye. They 
concluded that spectacles could only be 
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a disturbing factor, and that one should 
not trust things seen through lenses. 
The emission theory of vision had, in 
fact, been disproved by Ibn al-Haytham 
in the 11th century by consideration of 
after-images in the eye and the pain that 
results from looking at the sun. 

. Amazingly, about 50% of U.S. col- 
lege students still believed in visual 
emissions in 2002! See: Winer, G. A., 


Cottrell, J. E., Gtegg, V., Fournier, J. S., 
and Bica, L. A. (2002). Fundamentally 
Misunderstanding Visual Perception. 
American Psychologist, 57(6-7), pp. 
417-424. This paper reports that roughly 
one half of college students believe that 
vision depends on emanations from the 
eyes and that traditional education fails 
to overcome this belief. 


CHAPTER 8 


How to Make Lenses that Are 
Good Enough for Photography 


Nevertheless, it remains true that the lens is generally 
the most expensive and least understood part of any camera. 


—RUDOLF KINGSLAKE 


Every defect and aberration that a lens has will be 
revealed in an astrophoto. These include spherical 
aberrations, chromatic aberrations, coma, 
astigmatism, curvature of field, 
and vignetting. 


—JERRY LODRIGUSS 


8.1 Introduction 


fter photography was invented in 
A= there was a pressing need for 

lenses that could produce sharp, 
undistorted images with wide fields of 
view. The need was much less obvious when 
objects were viewed through the center of 
telescopic lenses and artists squinted at 
faint images on ground glass in a camera 
obscura, but photography provides perma- 
nent images that can be examined critically 
in bright light. The new requirements seem 
straightforward. We want all the light from 
an object point to be focused in the correct 
position on the film/sensor, regardless of the 
color. It turns out to be possible to make an 


acceptable image but not a perfect image. 
Here is the problem: We are limited by the 
laws of physics, the properties of available 
optical glass, and the technology for shaping 
glass surfaces. In this chapter we consider 
why multi-element lenses are necessary in 
order to minimize aberrations while provid- 
ing large apertures. We will also find that 
multi-element lenses give unacceptable 
light transmission, ghosts, and flare in the 
absence of antireflective multi-coating. 

All the possible types of spherical lenses 
are shown in Figure 8.1. As discussed in 
Chapter 6, those lenses that are thicker 
in the center (positive lenses) can be used 
to focus light, while the concave lenses 
(negative lenses) cause light rays to diverge. 
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Convex- 
concave 


Plano- 
convex 


Biconvex 


Plano- 
concave 


Meniscus Biconcave 


FIGURE 8.1. Simple spherical lenses. 


FIGURE 8.2. Photograph illustrating the effect of image curvature (simulated). 


Suppose we select a single biconvex lens 
for our camera. The resulting image will 
be something like the one illustrated in 
Figure 8.2. It may be possible to get a sharp 
image in the center but the resolution will 
deteriorate away from the center. The main 
problem is that the image projected by the 
biconvex lens is not flat, but there are other 
aberrations as well. 


In 1812, the English scientist W. H. 
Wollaston discovered that a meniscus 
shaped lens, actually convex-concave, with 
its concave side to the front, can produce a 
much flatter image and, therefore, a much 
sharper photograph overall. Unfortunately, 
Wollaston's simple lens is not suitable for 
photography because it exhibits extreme 
chromatic aberration, i.e. it focuses blue 
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light at a different place on the optic axis 
than yellow light. 

The failure of biconvex and convex-con- 
cave lenses in photography is just an example 
of the host of problems that must be faced 
when attempting to improve the photo- 
graphic lens. Major problems arise because 
it is relatively easy to produce lenses with 
spherical surfaces by grinding and polish- 
ing, but spherical lenses are not optimal. In 
fact, the major monochromatic aberrations 
(spherical aberration, coma, astigmatism, 
distortion, and curvature of field) result from 
the inability of spherical lenses to produce 
perfect images. If one has to use spherical 
lenses, the correction of aberrations requires 
a compound lens with multiple elements. 

Science advances by erratic steps. 
At times inventions are followed by the 
development of a theory that leads to sub- 
stantial improvements. For example, the 
steam engine was followed by the theory 
of thermodynamics. Thermodynamics 
then showed how to make a more efficient 
steam engine. In electronics, the order was 
reversed. Maxwell’s unifying equations 
came first, and later Hertz produced radio 
waves. In optics, we find a mixed bag. The 
great theoreticians Gauss, Petzval, von 
Seidel and others developed the theory 
of lenses; but many lens makers muddled 
along hoping to find a great lens design by 
trial and error.’ Consequently, the develop- 
ment of good camera lenses was a very slow 
process. The problem is that aberrations 
cannot be corrected without using a num- 
ber of spherical lens elements. This means 
that a lens maker must adjust the radius of 
curvature of the surfaces of each lens ele- 
ment, the thickness of the elements, their 
separations, the position of the aperture 
(stop), and even the type of glass in each lens 
element. Without some guiding principles, 
the lens maker is lost. 
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Everything in photography involves com- 
promise. We can never make a perfect lens, 
but we can make lenses that are optimized 
for some purpose, taking into account size 
and cost restrictions. Therefore, lens imper- 
fections are still with us, albeit minimized 
by advances in computation and materials 
science; and it is instructive to study the 
criteria used to evaluate modern lenses. 
Basically, we expect the following: 


1. Sharp image over the entire frame— 
(low aberrations with high resolution 
and contrast) 


2. Undistorted image—(limited barrel 
and pin cushion distortion) 


3. No colored fringes around objects— 
(little chromatic aberration) 


4, Uniform brightness over the image— 
(little vignetting) 


5. Little scattered light and good light 
transmission—(low flare and light loss). 


Our task is to explain the terms in 
parentheses and describe remedies. The 
first three items are related to lens aber- 
rations, though they are measured in 
different ways and usually discussed sepa- 
rately. Correction requires the combination 
of a number of spherical elements, some 
of which have a different index of refrac- 
tion. The modern lens designer knows lens 
theory and has access to catalogs of suc- 
cessful designs from the past. A new lens 
design can be based on simple calculations 
or possibly on a previous successful design, 
but the task of optimizing all the elements 
is done by computer. A modern lens design 
program such as OSLO (Optics Software 
for Layout and Optimization) requires the 
user to enter information about the lens ele- 
ments with a specification of glass types and 
the parameters that are to be held constant. 
Various parameters such as the curvature of 
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lens elements are specified as variables. The 
program then determines all the optimum 
parameters (for minimizing aberrations) by 
tracing a large number of light rays through 
the lens system. With such programs, lens 
performance can be measured without hav- 
ing to construct a real lens. 

In lens design some things are impos- 
sible to do with a given set of materials. A 
very wide-angle lens with a large aperture 
will always have some image distortion, and 
the resolution will be better in the center 
(on-axis) than at the edges. Similarly, zoom 
lenses introduce a new set of compromises. 
Usually, there will be some barrel distortion 
(outward ballooning of the edges) at the 
shortest focal length and some pin cushion 
distortion (pinching in of the edges) at the 
longest focal length. The wider the zoom 
range, the more difficult it will be to main- 
tain high resolution and low distortion for 
all focal lengths. One should keep in mind 
that a lens designer must decide what com- 
promises to make given the limitations of 
weight, size, and cost. Users can now find 
lens reviews online and can decide what 
they are willing to pay for and which trade- 
offs they are willing to make. 


8.2 Aberrations 


he term aberration appears so fre- 

| quently in the discussion of lenses 
that it is worthwhile to be careful 

with its definition.’ It is necessary to recall 
that the refraction of light rays at each opti- 
cal surface is properly described by Snell’s 
law. For those rays that are close to the 
optical axis and make small angles with it, 
the Snell’s law calculations can be greatly 
simplified by replacing the trigonometric 
function, sin 0, with 0 itself. As we saw in 
Chapter 7, this substitution amounts to 
what is called paraxial or first-order theory, 


and the concept of the perfect lens where 
sin 0 = @ is satisfied. However, with practi- 
cal lenses this approximation is not justified 
and something better is required. In 1857, 
Ludwig von Seidel turned to the next best 
approximation of sin 0 when @ is still small, 
namely sin O = 0 — 6?/6; he worked out so- 
called third-order optics in order to see what 
changes result from the extra terms. His cal- 
culation, still an approximation, revealed 
five independent deviations from the per- 
fect lens behavior for monochromatic light. 
These deviations are known as Seidel aber- 
rations or, more simply, aberrations. With 
larger angles, more extreme effects appear 
that are not really independent, but third- 
order optics provides a vocabulary and a 
good starting point for lens design. For a 
complete listing of the aberrations with 
diagrams, please visit Wikipedia and the 
Melles Griot website. 

It is interesting to consider how spheri- 
cal aberrations can be corrected by using 
only spherical lens elements. Suppose that 
a hypothetical aberration free lens focuses 
incident light rays that are parallel to the 
optical axis at a distance f behind the lens. 
When longitudinal spherical aberration 
(LSA) is present, parallel rays off the opti- 
cal axis are focused at distances less than 
fand the marginal rays serve to define the 
LSA. If the light is monochromatic, and the 
spherical aberration is small (described by 
third-order theory), it can be shown that 
the resulting spot size for a light ray is given 
by kf/N? where k is the aberration coeffi- 
cient that must be determined or perhaps 
looked up in a catalog, fis the focal length 
of the lens including the sign (+/—), and N 
represents the F-number. Thus, in order to 
improve the optical performance, we need 
to add elements so that the total aberra- 
tion for the system can be minimized. For 
example, a positive lens f, > O can be used 
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in combination with a negative lens f, < 0 
with the f/f, ratio adjusted to cancel the 
aberration. If other aberrations such as 
astigmatism, coma, field curvature, and dis- 
tortion are present, still more lens elements 
are required. 

If all these aberrations result from the 
use of spherical lenses, why not just use 
aspheric lenses? That is a good idea because 
aspheric lenses with appropriate shapes 
can eliminate spherical aberrations and cut 
down on the number of elements required 
in compound lenses. Unfortunately, high 
quality aspheric lenses are very expensive to 
produce. Molded plastic and glass lenses are 
easy enough to manufacture, but their qual- 
ity is often low. Better, though much more 
expensive, aspheric lenses can be shaped 
by diamond turning with a computer con- 
trolled lathe. Another route, advanced by 
Tamron, involves applying optical resin 
that can be shaped to the surface of a lens. 
Even with high manufacturing expenses, 
many modern lenses make use of one or 


more aspheric elements. For example, the 
Canon 17-40 mm and 10-22 mm wide- 
angle zoom lenses each have three aspheric 
lens elements.* 

Color fringes, or chromatic aberration 
(CA), require some discussion of optical 
materials. The problem is that the refractive 
index of glass or any transparent mate- 
rial depends on the wavelength of light, a 
property known as dispersion. Figure 8.3 
illustrates the effect of dispersion on light 
rays by lens elements. The refractive index 
for blue light is higher than for red light, so 
blue rays are bent through larger angles as 
illustrated in Figure 8.3. 

One way to compensate for dispersion 
is to combine lens elements (Figure 8.3(b)) 
that have been carefully selected to cancel 
chromatic aberration at two well-separated 
wavelengths. The positive lens element is 
often crown glass (made from alkali-lime 
silicates) with relatively low refractive index 
and the negative element is flint glass (silica 
containing titanium dioxide or zirconium 


FIGURE 8.3. (a) The refraction of light by positive and negative lenses; (b) The combination of a 
positive lens with a negative lens having a higher index of refraction to produce an achromatic 


compound lens. 
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dioxide additives) with a higher refractive 
index. These elements can be cemented 
together to make a single achromatic lens 
that has much less variation of focal length 
over the selected wavelength range than is 
possible with a single glass element. 

An achromat is a lens that has been cor- 
rected so that it has the same focal length at 
two wavelengths, usually in the blue and red 
regions of the spectrum. Lenses that have 
been corrected, perhaps with three ele- 
ments, to give identical focal lengths at three 
wavelengths, and to provide some spherical 
correction as well, are called apochromats. 
However, the best color corrected lens is 
the superachromat, which gives perfect 
focus simultaneously at four wavelengths of 
light and often into the infrared region as 
well. This near perfect correction requires 
the use of expensive low dispersion optical 
materials. 


8.3 The Petzval Sum 


he great lens designer Rudolf 
Kingslake said, “The design of any 
photographic lens is dominated by 
a certain mathematical expression known 


Object 


as the Petzval sum” He was referring to the 
specification of the image curvature that is 
produced by any simple or compound lens. 
This curvature, illustrated in Figure 8.4, is 
such an obvious characteristic of a simple 
lens that some opticians do not even regard 
it as an aberration. 

This curvature is, in fact, a unique kind 
of aberration because it does not depend 
on the aperture or other characteristic 
distances in compound lenses. It does, how- 
ever, depend on the focusing powers of the 
optical surfaces, and in 1839 Joseph Petzval 
(1807-1891) derived an equation for the 
field curvature expected from a compound 
lens with a number of optical surfaces in 
the absence of other aberrations. He used 
the fact that the inverse of the image cur- 
vature (I/R, resulting from the i‘ surface 
of an optical element is proportional to the 
focusing power of the surface defined as P, = 
(1, —1,_,)/R, where n, and n, , are the refrac- 
tive index before and after the surface and 
R is the radius of curvature of the surface. 
Petzval showed that the curvature for the 
image plane resulting from all the lens sur- 
faces is given by the summation 


FIGURE 8.4. An illustration of image curvature. 
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(1/R,) = n, Dr / (a, n) 


where n, is the refractive index of the last 
medium.! When one says that the Petzval 
sum for a lens is large, they are just saying 
that the image curvature is large. Of course, 
the aim is to make the Petzval sum as close 
to zero as possible by a combination of posi- 
tive and negative surface powers. 


8.4 Optical Materials 


good material for a lens must not 
A us light in the visible region, 

but it also must have a refractive 
index greater than one — and the higher, 
the better. Of course, all materials absorb 
some light, and, if the thickness is great 
enough, the absorption will be evident. The 
normal situation in optics is for the refrac- 
tive index to increase as the wavelength of 
light decreases so that the index of blue 
light is greater than that of red light. This 
effect was modeled by the mathematician 
Augustin-Louis Cauchy in 1836, and in 
1871 his empirical equation was improved 
by W. Sellmeier to obtain the equation still 
used to characterize optical materials.° 
This equation relates the refractive index 
to the wavelength by means of empirically 
determined constants: 


ah, Bg) 
Al CO “eat,” 


Here n(1) indicates that n depends on 
the wavelength of light 1, and the B and 
C coefficients are used to describe optical 
glasses in the standard (Schott) catalog of 
glasses. 

Optics catalogs also characterize the dis- 
persion of materials by means of a standard 
parameter known as Abbe's number, v,.*”* 
This parameter is widely used although it is 


a rather crude measure of dispersion. The 
definition is 


(8.2) 


where n, n, and n are the indices of refrac- 
tion at the wavelengths of the Fraunhofer 
lines helium D (yellow, 589.2 nm), hydrogen 
F (blue, 486.1 nm), and hydrogen C (red, 
656.3 nm), respectively, that are found in 
the spectrum of the sun. In some treatments 
mercury (Hg) lines and cadmium (Cd) lines 
are substituted, but the wavelengths are not 
much different. In the definition of v, the 
numerator is a measure of the refractivity 
while the denominator is proportional to 
the dispersion.’ 

The most commonly used optical glasses 
are crown (v, ~ 60, n = 1.5) and flint (v, < 
55, n = 1.7). The problem is that the disper- 
sion tends to be large for glasses that have 
large refractivities. This is evident from 
the Sellmeier equation. Absorption wave- 
lengths are usually in the UV part of the 
spectrum; and, when an absorption wave- 
length is close to the visible region, both the 
value of the refractive index and the rate of 
change of the index with wavelength tend 
to be large. The Schott glass company has 
published plots of refractive index versus 
the Abbe number so that lens designers can 
select appropriate glasses. Basically, one can 
easily get high refractivity or low dispersion, 
but not both. 

As with the problem of spherical lens 
shapes, one must live with the limitations 
imposed by nature. As previously noted, 
one way to minimize color fringing (chro- 
matic aberration) is to play off different 
kinds of glass against each other. Another 
possibility is to search for natural and syn- 
thetic materials that push the limits of 
the Sellmeier equation by providing low 
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dispersion with acceptable refractive indi- 
ces. This could result from a fortunate set of 
absorption frequencies. For example, crys- 
tals of fluorite (CaF,), which has v,,= 95 and 
n, = 1.43, occur naturally but can also be 
grown for use in lenses. Ernst Abbe was the 
first person to use fluorite to enhance chro- 
matic correction in microscope lenses in 
the 19th century. Fluorite is unfortunately 
difficult to work with, and its refractive 
index is so low that lens elements must 
be thick and heavy. So lens makers have 
turned to proprietary engineered glasses 
that are claimed to have low dispersion and 
fairly high refractive indices. For exam- 
ple, we find Hi-UD (Canon), ED (Nikon), 
AD and LD (Tamron), and SLD (Sigma) 
low dispersion glasses advertised for new 
lenses. 

A discussion of corrections for chro- 
matic aberration would not be complete 
without mentioning diffractive optics. The 
diffraction effect from a concentric lattice of 
transparent plates, as in a Fresnel lens, dis- 
perses red light more than blue — exactly the 
opposite of a glass prism or lens. So the idea 
is to use both diffractive and dispersive lens 
elements in a compound lens to compensate 
for chromatic and possibly other aberra- 
tions. Thus a lens element is introduced that 
has a diffraction grating etched into it. Early 
work in this area was delayed by problems 
with excessive diffraction based flare, but 
now Canon has developed multilayer dif- 
fraction optics, or Diffractive Optics (DO), 
that are competitive at least for telephoto 
lenses. The primary advantage appears to 
be more compact and lighter lenses. For 
example, the Canon 400 mm F/4 DO IS 
lens is claimed to be reduced 37% in length 
and 31% in weight from a comparable lens 
designed only with refractive optical ele- 
ments. Nikon has a competitive technology 
known as Phase Fresnel (PF) that appears to 


be similar to diffraction optics. At this time, 
only Canon and Nikon offer lenses incorpo- 
rating diffraction or Fresnel optics. Now, on 
the horizon we see thin optical components 
that promise thin lenses with no chromatic 
aberrations. This depends on an engineered 
wavelength-dependent phase shift provided 
by an achromat metasurface (see Section 
IT) 


8.5 Anti-Reflective Coatings 


that compound lenses incorporating 

many elements are required to correct 
the various aberrations; and commercially 
available modern lenses bear witness to 
this requirement. Samples from lens cata- 
logs show lenses with large numbers of 
elements. Also, low dispersion glass and 
aspheric elements are used in crucial posi- 
tions to improve performance. For example: 


E turns out, with current technology, 


Canon EF 100-400 mm f/4.5-5.6 L IS II, 
21 elements in 16 groups (fluorite and 
Super UD-glass elements) 


Sigma 120-300 mm f/2.8 DG OS HSM | 
S, 23 elements in 18 groups (SLD and 
FLD elements) 


Nikon 80-400 mm f/4.5-5.6 ED AF VR 
AF-S Nikkor, 20 elements in 12 groups 
(4 ED and 1 super ED element) 


The modern multi-element lens is only 
possible because of a technological break- 
through that I have not yet discussed. In the 
19th century, lenses with more than four air/ 
glass surfaces suffered from low contrast. 
Some lenses with six glass/air surfaces were 
considered acceptable; but the eight air/ 
glass surface lenses, while better corrected 
for aberrations, were found to yield images 
with very low contrast. The problem was 
the reflection of light at the surfaces. When 
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light passes from a region with the refractive 
index n, to a region with the refractive index 
n, a fraction R of the intensity is reflected. 
Equations for R were worked in the early 
19th century by Augustin-Jean Fresnel. In 
the limit that the incident light is normal to 
the surface, he found that: 


r= E a al Rer 
n, +n, 

Here r and R are associated with the ampli- 
tude and the intensity of the reflected light, 
respectively. (The amplitude r refers to the 
electric field of light, and it will be needed 
to explain the effect of lens coatings.) For 
example, at an air/glass interface with n,= 1 
and n,= 1.5, the fraction of reflected inten- 
sity is R = 0.04 or about 4%. This fraction is 
lost at every air/glass surface, so with four 
surfaces about 15% is lost and only about 


(8.3) 


85% of the incident intensity is transmitted. 
This was, indeed, a serious and disabling 
problem for lens technology. Light was not 
only lost, but the reflected light bounced 
around inside the lens, producing ghosts, 
flare, and a general loss of contrast in the 
focused image. 

The solution to the problem lies in sophis- 
ticated anti-reflection coatings. Everyone 
with a camera has, of course, observed the 
beautiful violet and pastel shades that are 
reflected by several elements in the lens. The 
reflections from lenses manufactured in dif- 
ferent time periods are shown in Figure 8.5. 

The colors reveal coatings that are them- 
selves colorless. Nevertheless, these coatings 
increase the transmission of a broad range 
of frequencies in the visible spectrum while 
reflecting a small amount of light that has not 
been completely cancelled by interference. 
The discovery of the effects of lens coatings 


ies 4 


FIGURE 8.5. Reflections of a 5000K lamp from various lenses. (a) Olympus 50 mm f/1.8 (1972), 
(b) Canon 70-300 mm f/4-5.6 (2006), (c) Canon 50 mm f/1.8 (1987), and (d) Schneider-Kreuznach 


75 mm f/3.5 (1953). 
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has an interesting history. Isaac Newton 
observed the “colours of thin transparent 
bodies” in 1672-1675 and reported his find- 
ing in the second book of Opticks (1704). He 
knew about missing reflections associated 
with half-wavelengths, but his ideas are very 
hard to follow, and there were no practical 
applications. However, in the nineteenth 
century, with the invention of photography 
and advances in optics, the time was right 
for anti-reflective optical coatings. The early 
days of optical coating has been reviewed 
in a delightful article by Angus Macleod.” 
The beneficial anti-reflecting property 
of coatings was accidentally discovered 
several times by famous scientists. For 
example, in the early 19th century Joseph 
von Fraunhofer observed the reduction of 
reflection from glass that was tarnished by 
acid treatment. Lord Rayleigh (1886) noted 
that old lenses (aged glass) transmitted light 
better than new (clean) lenses, and sug- 
gested the presence of a surface layer with 
a lower refractive index than the glass. So 
the idea that “age coated” as well as inten- 
tionally tarnished lenses were preferable 
to those with bare air/glass surfaces was 
floating around by the 1890s. Apparently, 
the first person to attempt the commercial 
use of coated glass was Dennis Taylor, who 
patented the use of various chemicals to 
deliberately tarnish optical surfaces in 1904. 
Unfortunately, Taylor's results were incon- 
sistent, and coating development lagged 
until the 1930s. 

The next breakthrough occurred in 1935 
in Germany when Alexander Smakula at 
the Zeiss company developed a method for 
coating lens surfaces in a vacuum with an 
evaporated layer of low index material. At 
about that time optics research became part 
of the war effort, and developments were 
classified. By 1941 the transmission of light 
by multi-element lenses had been improved 


by about 60%; and anti-reflection coatings 
were being ordered for periscope and bin- 
ocular lenses. Macleod notes that coated 
binoculars permitted an extra 30-45 min- 
utes of viewing time at dusk and dawn. Also, 
multi-layer coatings were being developed 
in Germany, and by the end of the war opti- 
cal coating was a big business. 

Since World War II, multilayer coatings 
have been developed to both enhance and 
diminish reflection, select wavelengths for 
transmission or reflection, polarize light, 
split beams, etc. A major driving force for 
the improvement of optical coating tech- 
nology has been the development of lasers. 
Gas lasers, e.g. the He-Ne laser, especially 
require highly reflective multilayer coatings 
for mirrors. Of course, photographers bene- 
fit greatly from the availability of multilayer 
coated lenses. To quote Rudolf Kingslake, 
“The discovery of antireflection coating of 
lenses opened up the whole field of multi- 
element optical systems, so that now almost 
any desired number of air spaced elements 
can be used without danger of ghosts and 
flare. Without such coating all of the mod- 
ern zoom and other complex lenses would 
be impossible” 

Now for the science.!? How do anti- 
reflection coatings work? Suppose a thin 
transparent coat with a refractive index 
of n, = 1.25 is applied to a glass lens that 
has the refractive index n, = 1.5. According 
to Equation (8.3), the fraction of intensity 
reflected at the air/coat surface is 1.2%, and 
the fraction reflected at the coat/glass sur- 
face is 0.83%. Now the important question 
is how to obtain the total transmission, and 
the numbers we have calculated are not 
helpful at all! Experiments show that the 
total reflection depends critically on the 
thickness of the layer. Transmission and 
reflection from thin layers are interference 
phenomena, and amplitudes of the electric 
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FIGURE 8.6. Single layer anti-reflective coating with exaggerated angle of incidence. 


fields must be combined before squar- 
ing. The amplitude of a reflected wave, 
which should be thought of as the length of 
an arrow (vector), is given by r (not R) in 
Equation (8.3) and each arrow has an ori- 
entation. This idea is discussed in Appendix 
B. A single layer anti-reflective coating 
is illustrated in Figure 8.6. (The arrows in 
this figure indicate the directions of light 
rays and the associated amplitude arrows 
denoted by the r's are different things.) 

Suppose that n, = 1 (air) and n, = 1.5 
(glass). The coating layer must have a refrac- 
tive index less than 1.5 (for there to be the 
same change in phase angle or turn of the 
amplitude arrow at the two surfaces); and, 
for effective cancellation of reflection, it is 
necessary that r, = r, and that their orien- 
tation be exactly opposite. This condition 
can be combined with the equation for r to 
show that the optimum refractive index for 
the layer is n, = Vn, Ny» or in this case 1.22. 
We have a situation in which there are two 
ways for a photon to get from the source to 
the observer, and according to the ideas in 
Appendix B each of the paths can be rep- 
resented by an arrow with a length and an 
angle. These arrows must be determined, 
added together, and the resultant arrow 
length squared in order to determine the 
intensity of the reflected light. 


The lengths of the arrows are essentially 
equal because of the choice of n,, and the dif- 
ference in orientation (angles) depends only 
on the difference in the optical path lengths. 
The orientation of the arrow associated with 
r (path 1) is arbitrarily set at 12 o'clock, and 
the orientation of the arrow for path 2 is 
different only because of the time required 
for the photon to traverse the layer of thick- 
ness d, twice with the effective speed c/n,. A 
complete rotation of the arrow through 360° 
occurs each time the path length increases 
by one wavelength; and with our choice of 
layer thickness, the arrow for path 2 is ori- 
ented at 6 o'clock or 180° out of phase with 
the arrow for path 1. The two arrows, when 
placed head to tail, add to zero; and there is 
complete cancellation of the reflected light 
at the wavelength >. Since the sum of the 
intensities of the reflected and transmitted 
light must equal the intensity of the incident 
light, this coating permits 100% of light at A, 
to be transmitted. I have simplified things 
here by assuming that the angle of inci- 
dence is close to zero, the amplitudes of 
reflection are very small, and by neglecting 
multiple reflections in the layer. Corrections 
for these simplifications do not change the 
qualitative picture. 

Of course, things are far from perfect, 
because the single layer is only efficient at 
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cancelling reflection at one wavelength. With 
a two-layer coating it is possible to zero out 
reflection at two wavelengths, a three-layer 
coating can be designed to cancel reflection 
at three wavelengths, etc. Approximate anal- 
yses of these situations are quite easy with 
the vector method summarized here. 

However, there is still a serious prob- 
lem, because multi-coated surfaces begin 
to lose their effectiveness for light at large 
angles of incidence. Accordingly, reflec- 
tions from high curvature lenses produce 
flare and ghosting. Reflections from the 
surface of sensors also contribute to this 
problem. One way to ameliorate this prob- 
lem is to turn to lens coatings that mimic 
the surface of the corneas of some insect 
eyes. In particular, the cornea of a moth eye 
shows low reflectivity because of patterned 
microstructures on its surface. The idea is 
that subwavelength structures with cone or 
pyramid shapes can form a layer with much 
lower average density at its surface than at 
its base. One can picture a forest of tapered 
needles. This provides a refractive index 
gradient ranging from perhaps 1.0 to 1.5 
that gives a smooth transition from air to a 
glass lens element.” 

Subwavelength structure coatings 
(SWC) are now available on some lenses 
from Canon, Inc. The coating of large 
curved surfaces with microcrystalline alu- 
mina (A1,O,) films is possible through the 
use of sol-gel solutions. After drying, the 
films are random and complex; but they 
have the necessary thickness and proper- 
ties. If the maximum refractive index does 
not equal that of a glass lens element, an 
intermediate layer may be necessary. Tests 
show that SWC lenses have lower reflec- 
tivity over a wider wavelength range than 
conventional multi-coated lenses. Also, 
ghosting is largely eliminated in demonstra- 
tion photographs. 


8.6 Conclusion 


set out to show why modern lenses are 

so complicated. That should be evident 

by this point. But, it should be noted 
that the discussion thus far applies to fairly 
simple “prime” or non-zoom lenses. Also, 
most modern lenses contain autofocus 
mechanisms and in some cases vibration 
isolation systems based on the movement of 
a lens element to counteract low frequency 
vibrations. These important technological 
refinements are beyond the scope of this 
book, but some of the optical characteristics 
of compound lenses, e.g. primary surfaces 
and nodal points, will be considered in later 
chapters. 
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1892 (Drude-Lorentz theory) in terms 
of oscillating electrons in the electric 
field of light rays. Of course, our view 
of light and matter changed com- 
pletely with the discovery of quantum 
mechanics. The translation of Lorentz’s 
equation into quantum language was 
performed by John van Vleck (1925) 
and by Hans Kramers and Werner 
Heisenberg (1926). The latter paper, 
co-authored by Heisenberg, is credited 
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quantum mechanics (matrix mechan- 
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thereafter. The modern interpretation of 
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tion n(A) on wavelength 1 results from 
absorption of light at the wavelengths 
A, = VC, A, = VCs and A, = VC, The 
contributions to (A) of the absorptions 
at À» A, and A, are determined by the 
coefficients B, B,, and B, respectively. 
These coefficients are related to the 
strengths of the absorptions and can, in 
principle, be computed by the methods 
of quantum mechanics. Each absorp- 
tion is associated with the transition of 
an electron from its ground energy state 
to a higher energy state. For example, 
in the transition at A, a photon of light 
with energy hv, is absorbed by an elec- 
tron to produce the energy change AE = 
hy, = Eaa = Eee, 
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CHAPTER 9 


Coming to Terms with 
Real Camera Lenses 


Any sufficiently advanced technology 
is indistinguishable from magic. 


—ARTHUR C. CLARKE 


9.1 Introduction 


e have seen why compound, 
multi-coated photographic 
lenses are needed to produce 


aberration free images with high contrast, 
but we have not considered how to deal 
with camera lenses that often look like pipes 
with glass at each end. The overall length 
of camera lenses is often greater than their 
effective focal length, and it is not obvious 
which parameters are significant. Indeed, 
some extreme telephoto lenses are truly 
“monster lenses.” For example, the Canon 
400mm, f/2.8L IS II lens is 13.5 in. long and 
weighs 8.49 lb; and it is not even the heavi- 
est lens available for a 35 mm (FF) camera. 
It is natural to ask how we can get away 
with discussing focal length, field of view, 
depth-of-field, etc., in terms of single ele- 
ment lenses when real lenses look nothing 
like that. Fortunately, simple lens ideas and 
equations are still useful and provide good 


approximations to compound lens behavior 
(as long as objects are not too close to the 
lens), but there is a lot more to understand. 

Clearly, photographers do not need to 
know all the internal details of the lenses 
they use, but it is satisfying to understand 
the primary characteristics of the beautiful 
modern lenses that are available to us. Also, 
there are situations where a user needs some 
information about lens parameters—for 
example, panoramic pictures that are con- 
structed by stitching together overlapping 
images that often do not match properly for 
nearby objects. The problem is that proper 
alignment of objects in the images requires 
the camera to be rotated around the proper 
axis. This “no-parallax axis” must be deter- 
mined either by doing experiments or by 
locating the entrance pupil of the lens. 
(This assumes that the sensor rotates with 
the lens.) The goal of this chapter is to 
explain how camera lenses behave without 
a detailed consideration of all the internal 
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components. For the most part we can get 
by with a “black box” approach. 

Let’s begin by trying to imagine a perfect 
simple lens. This is not as straightforward 
an exercise as one might think. If technol- 
ogy and materials pose no limitations, what 
do we want in a lens? First, rays of light from 
points on an object must pass through the 
front surface of the lens and then emerge 
from the rear surface directed at precisely 
the correct points in the image plane. All 
of the rays from a properly focused object 
point must reach the image point at exactly 
the same time so that they are in phase and 
contribute to the intensity. Furthermore, 
perfect focus must occur for all colors in 
order to form an aberration free image. This 
perfect lens may have aspheric surfaces and 
is fabricated from a wonder material that 
has a high refractive index but no dispersion. 
But wait, modern lenses, while not perfect, 
do more than that. We expect variable focal 
length (zoom), automated focus and aper- 
ture, and even image stabilization. I can get 
most of that out of the way for the perfect 
lens by assuming that the material proper- 
ties, e.g. the value and spatial distribution 
of the refractive index, can be controlled 
externally by electric and magnetic fields 
or by engineered metasurfaces. For present 
purposes, I will concentrate on static opti- 
cal properties; but I note that the surfaces 
do not have to have any particular shape 
since the optical path length can, in prin- 
ciple, be controlled with internal variation 
in the refractive index. 

So the perfect lens is nothing more than 
two surfaces with something between them 
that transforms light rays incident on the 
front surface into rays that exit the rear 
surface with the correct phases and direc- 
tions. With current technology we can only 
approximate the optical properties of the 
perfect lens. That is to say, our real camera 


lens will have some kind of entrance and exit 
surfaces and between them will be a “magic 
zone” or black box containing as many lens 
elements with various refractive indices 
as are necessary to transform the incident 
rays into the desired exit rays. This picture 
of real lenses is a good starting point for 
relating simple lenses to compound lenses, 
and I will begin by reviewing the way simple 
lenses behave. 


9.2 The Thin Lens 


thin lens, as shown in Figure 9.1, has 

an axial thickness that is much less 

than the lens diameter and the focal 
length; and distances along the axis can be 
measured from the central plane of the lens. 
Rays, from objects to the left, that are paral- 
lel to the optical axis, after refraction, are 
focused at F,, the rear principal focus point 
For closer objects, a real image will always 
be formed at a distance greater than f from 
the lens. The imaging properties of this lens 
can be specified with a short set of rules: 


1. Rays passing through the center of a 
thin lens are undeviated. 


2. Rays parallel to the optical axis pass, 
after refraction, through the focal 
point (F,) of the lens. 


Of course, this works in both directions 
so a ray passing through a focal point will, 
after refraction, emerge parallel to the opti- 
cal axis. These rules suffice for determining 
the magnification m = image/object = q/p 
and also for deriving the conjugate equation 
1/p + 1/q = 1/f. With this lens there is no 
ambiguity. The distances are well defined; 
and, if a stop (aperture) is added, it will 
certainly be very close to the front or rear 
surface of the lens. 
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FIGURE 9.1. Image creation with a simple lens. 
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9.3 The Compound Lens 


or the compound lens we begin 
Fi an approximate picture, based 

on first-order optics. The thick lens 
shown in Figure 9.2 with its cardinal points 
(principal and nodal points) and front and 
rear focal points is an idealization of a real, 
compound lens. It is accurate for paraxial 
rays, i.e. rays near the optical axis and mak- 
ing small angles with that axis. 

The vertical lines P, and P, represent 
the first and second principal planes and 
N, and N, are the corresponding nodal 
points. Assuming that the refractive indi- 
ces are the same in the object and image 


spaces, the nodal points lie in the principal 
planes; and the front and real focal lengths, 
f, are equal. We do not have to specify what 
occupies the Nodal Zone or “hiatus,” but 
only the nature of the transformation it 
produces. The rays do not follow the exten- 
sions shown in the Nodal Space, but act as if 
all deviations occur at the principal planes. 
We can describe the behavior of the com- 
pound lens with a short (incomplete) list of 
rules describing its effect on incident rays 
lying in a plane containing the optical axis 
(meridional rays): 

1. Rays passing through P, parallel to the 


axis also pass through the focal point 
F, on the opposite side. 


Magic Zone 
(Nodal Space) 


FIGURE 9.2. Image formation by a compound lens. 
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Sensor 


FIGURE 9.3. The field of view for a simple lens. 


2. A ray passing through N, from the left 
emerges from N, at the same angle 
with the axis. 


3. A ray entering P, at a height / above 
the axis exits P, at the same height 
from the axis. 


This thick lens acts as if there are two 
refracting surfaces, and the distance from 
the principal plane P, to the focal point 
is called the focal length. The conjugate 
equation can also be carried over from the 
simple lens by measuring p and q from the 
principal planes rather than from center of 
the lens. Also, the field of view illustrated in 
Figure 9.3 can be calculated with the simple 
lens equation FOV = 20 = 2tan-! (S/2f). Here 
S is a sensor dimension and can be defined 
as the width (horizontal), height (vertical), 
or the diagonal. 

Before becoming too comfortable with 
the thick lens as illustrated, one must realize 
that the principal planes are mathematical 
constructions. They can in principle have 
any separation and, in fact, P, can be located 
in front of P.. 


9.4 Gaussian Optics 


o howcan this simple picture based on 
the cardinal points be justified? What 
I need to show is that the equations 


that describe a simple, thin image-forming 
lens also describe the behavior of a com- 
pound lens. To do this, I turn to the theory 
known as Gaussian optics that is attributed 
to Carl Friedrich Gauss (1777-1855).*?* 
This theory applies to light rays that remain 
close to the optical axis. (Their angles with 
the axis are small, and when they intersect 
an optical surface they make a small angle 
with the surface normal. This is just the con- 
dition for first order optics to hold where 
for any angle 6, sin @ can be replaced with 
0 itself.) The compound lens, no matter 
how complicated, just consists of spherical 
surfaces separating regions with constant 
refractive indices. The “power” of a sur- 
face is given by (1, — n,)/R, which is just the 
change in the index of refraction on passing 
through the surface divided by the radius 
of curvature of the surface. In a meridional 
plane, i.e. a plane that includes the opti- 
cal axis, each optical surface appears as a 
segment of a circle with its center on the 
axis. 

If the math is getting scary, hang on, 
because the ideas are really very simple. The 
situation is illustrated in Figure 9.4. At any 
time a ray is described by just two things: 
its distance (x) from the axis and the angle 
it makes with the axis. 

As the ray passes through a compound 
lens it can be transformed in two ways: 
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(a) 


FIGURE 9.4. (a) A ray passes through surface A and the angle changes from 6, to 6’; (b) A ray 
passes through a simple lens consisting of two surfaces, A, and A,, and the space between them. 


1. The ray passes through a surface and 
its angle with the axis changes but not 
its distance from the axis. 


2. The ray translates and its distance 
from the axis changes but not its angle 
with the axis. 


The effect of a single surface is shown in 
Figure 9.4(a) where the angle changes but 
the height does not. The combined effect 
of a surface, a space, and another surface 
is illustrated in Figure 9.4(b). The effect of 
a surface depends on its power, while the 
effect of a translation only depends on the 
angle of the ray and the distance traversed. 

In short, here is the story. One can work 
out, in a straightforward way, the effect on 
a light ray of all the optical surfaces and the 
spaces from the first surface to the last sur- 
face. The mathematical description of this 
transformation can then be compared with 
the transformation produced by a single 
thin lens. It turns out that the two descrip- 
tions are identical in form, if and only if, the 
computation for the compound lens begins 
and ends at the proper principal planes. 
Locating these planes is also straightfor- 
ward, but there is no requirement that 
they be located close to real surfaces and, 
as noted above, the second principal plane 
may appear to the left of the first principal 


plane. The complete details can be found in 
standard optics texts. Also, an outline of the 
theory is given in Appendix D. 

Gaussian optics provides a good descrip- 
tion for rays close to the axis, and can 
provide a starting point for designing com- 
pound lenses, but, of course, ray tracing 
is required for designing modern photo- 
graphic lenses. One place that Gaussian 
optics is still very useful is in setting up 
systems of lenses to direct laser beams. The 
beams of nearly parallel light rays, i.e. col- 
limated beams, are usually close to the axis 
and furthermore are monochromatic. 


9.5 Entrance Pupil, Stop, and 
Illumination of the Image 


t this point I want to consider the 

factors that determine the illumina- 

tion of the image.° The brightness of 

the object is the first factor. Light from the 

object O is radiated into the cone defined 

by the half-angle 0, and a fraction of that 

light is transmitted by the optical system to 

the image I. The image is illuminated by a 

cone of rays defined by the half-angle 6’as 
illustrated in Figure 9.5. 

The important fact is that the illumi- 

nation of the image is proportional to the 

brightness B of the object and the square of 
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FIGURE 9.5. The object O, the optical system, and the illuminated image |. 


FIGURE 9.6. The spherical refracting surface and the cone of light illuminating an image at E. 


sin 0”, i.e. the illumination is proportional to 
B(sin 0”. Figure 9.6 illustrates the image side of 
the compound lens, and correctly shows that 
the second principal “plane” in a corrected 
lens is, in fact, a refracting spherical surface. 

When distant objects are photographed 
it is appropriate to replace sin 6’ with r/f 
This substitution motivates the definition of 
the F-number of the lens: 


1 f 


Qsing’ 2r ` 


F-number = N = 


Here the diameter 2r refers to the “effec- 
tive aperture” of the lens. This is not the 
diameter of the front lens element or even 
the iris diaphragm. The “effective aper- 
ture” or entrance pupil is the image of the 
“aperture stop” as seen from the front of 
the lens. What we mean by “stop” is a phys- 
ical aperture that limits the cross-section 
of the image-forming pencil of light. The 
stop is commonly an opaque sheet with an 


adjustable (iris) aperture, but it may be the 
edge of a lens or a diaphragm. The aperture 
stop can be anywhere in the compound lens 
or even outside the lens. In lenses that have 
been corrected for spherical aberration and 
coma the equivalent refracting surface is 
part of a sphere. This sphere is centered on 
the focal point, and a simple construction 
as in Figure 9.6 shows that the radius r can 
never exceed f. The conclusion is that the 
theoretical maximum aperture corresponds 
to N = 0.5. 

The important point is that we see the 
effective aperture when we look through a 
lens from the front. It is this apparent diam- 
eter that must be used for computing the 
F-number and the depth-of-field. Figure 9.7 
shows the effective apertures for two lenses. 

The entrance pupil of the lens on the left 
is very close to the front of the lens, while 
the entrance pupil for the lens on the right is 
much farther back. Also, the apparent loca- 
tion of the effective aperture moves as the 
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focal length of the zoom lens is changed. It 
is the apparent location of the stop that is 
the pivot point or “no-parallax” axis of the 
lens.* The entrance pupil is also the per- 
spective point of the lens. Fortunately, the 
location of some “no-parallax” points are 
published online.” 


9.6 Vignetting 


he loss of intensity at the edges and 

particularly the corners of an image 

is known as vignetting.? This effect 
can be desirable for removing distractions 
and drawing attention to the center of a pho- 
tograph. However, unintentional vignetting 
is a defect that needs to be minimized or 
removed. There are at least three sources 
of vignetting. The most obvious one results 
from obstruction of oblique rays by features 
in the lens barrel or lens attachments such 
as filters or hoods. I will refer to both inter- 
nal and external “cut-offs” as mechanical 
vignetting, although some authors prefer 
optical vignetting for the effects of internal 
obstructions. From Figure 9.7 it is clear that 
internal blockage will be most pronounced 
with large apertures and long lenses. An 
extreme example of mechanical vignetting 
resulting from the use of an inappropri- 
ate lens hood on a 28 mm lens is shown in 


Figure 9.8. External obstructions can, of 
course, be removed, but we are stuck with 
the obstructions resulting from lens design. 
However, Photoshop and other software 
packages use lens profiles to correct for 
vignetting and lens aberrations. 

There is variation of illumination of 
images over the field of view even in the 
absence of mechanical vignetting. This 
natural vignetting is geometric in origin 
and is approximated by the Cos* Law. The 
following is a simplified derivation of the 
Cos* Law based on Figure 9.9. (It is instruc- 
tive and reasonably good for small aperture 
lenses.) The illuminance of an area on the 
sensor at angle q is reduced relative to 
the illuminance at angle zero by four factors 
of cos q (as defined in Figure 9.9). 

The first cos q factor results from the 
oblique angle at which the circular aperture 
is viewed. For the off-set detectors at angle 
q, the aperture appears to be an ellipse with 
its height and area reduced by a factor of cos 
q. The second factor arises because a beam 
of light striking the sensor at ¢ is spread over 
a larger area of the sensor than a beam nor- 
mal to the sensor. Finally, the illuminance is 
inversely proportional to the square of the 
distance from the aperture. The ratio of the 
distances to the sensor at angle q, and angle 
zero is C/A or 1/cos q, and the inverse of 


FIGURE 9.7. The entrance pupils of two lenses. 
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FIGURE 9.8. Example of mechanical vignetting caused by a lens hood. 


Aperture 


Sensor 
E, 
cos $ = A/C 
AL | 
E, A 


FIGURE 9.9. Geometric factors determining illumination of detectors in sensor. 


this quantity squared gives the required fac- 
tor of cos? q. Thus we have found that the 
illuminance at angle q is reduced by a factor 
of cost q, and this result has been derived 
by using only the geometry on the image 
side of the lens. For some lens designs it is 
preferable to determine the first cosine fac- 
tor from the object side, where the apparent 
shape of the entrance pupil also depends on 
the angle of the oblique ray. This approach 
will give a different angle in the first cosine 
factor if the lens has been specially designed 
to increase the aperture to sensor distance. 
Such retrofocus lenses are often found in 


DSLRs where space must be available for 
the mirror between the lens and the sensor. 

With digital cameras there may be yet 
another mechanism for intensity loss at the 
edges of the image. As illustrated in Figure 
9.9, the detectors do not lie on the surface 
of the sensor but rather at some depth in the 
chip, effectively down “tunnels” In CMOS 
sensors they occupy the pixel site along 
with other circuit elements, and it is com- 
mon for the pixel to be covered by a small 
lens that directs light down the “optical tun- 
nel” to the photodiode. The problem is that 
oblique rays are not as efficiently captured 
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by the pixel lens as are rays normal to the 
sensor. The resulting loss of intensity has 
been called pixel vignetting. The problem is 
less severe with CCD detectors where active 
transistors at the pixel site are not required 
for readout, and the entire pixel can be light 
sensitive. In this case the pixels are defined 
by thin gates. CCD sensors have the advan- 
tage of efficient light collection, but the 
advantage is not significant for large pixels, 
i.e. 10um or greater. CMOS sensors now 
dominate the digital camera world, and it 
is fortunate that recent back-illuminated 
designs greatly reduce the pixel vignetting 
problem. 

Software correction for vignetting is, 
of course, available in programs such as 
Photoshop (PS) and Lightroom. PS provides 
a slider control for increasing or decreasing 
brightness in the corners and a midpoint 
slider for setting the area affected by the 
correction. Software correction is effec- 
tive for all types of vignetting, including the 
fall-off of flash illumination away from the 
center that is common when wide-angle 
lenses are used. 


9.7 Metalenses and the New 
Optics of the 21st Century 


his chapter concludes an introduc- 

tion to optics and optical materials 

emphasizing photographic applica- 
tions. Conventional optics, as described 
here, is based on the application of Snell’s 
law at interfaces between regions of differ- 
ing refractive indices and propagation in 
regions where the refractive index is con- 
stant. Thus, changes in phase accumulate 
through propagation along paths much 
longer than the wavelength of light. These 
ideas are relevant for available optical 
components. However, advances in meta- 
materials (synthetic composite materials) 


and metasurfaces (planar arrays of sub- 
wavelength spaced phase shifters) are now 
driving disruptive technologies that prom- 
ise unprecedented improvements in the 
performance of lenses and other optical 
components, as well as lower costs.*? 

In contrast to conventional lenses, it is 
now possible to fabricate thin, flat lenses 
that can focus light without spherical aber- 
rations. These metalenses function by 
introducing abrupt phase and polariza- 
tion changes as a function of position on 
the surface. The metalens demonstrated 
by Khorsaninejad, et al.,? makes use of an 
array of TiO, antennas (scatterers) with 
subwavelength spacing and height. These 
remarkable nanostructures or fins are 
produced by electron beam lithography. 

Some of the properties of thin meta- 
lenses can be glimpsed by considering a 
surface lying in the xy-plane that contrib- 
utes position-dependent phase shifts, f(x,y), 
to light rays normal to the surface. Now 
suppose that there is a constant phase shift 
gradient, df/dx, so that the effective optical 
path length associated with traversing the 
surface changes with position. In this situ- 
ation, it is necessary to generalize Snell’s 
Law. In Chapter 6 we encountered Fermat’s 
principle of least time, namely that light 
rays go from a source to a point of focus in 
the path requiring the least time. For light 
rays propagating in media with different 
indices of refraction, the lifeguard analogy 
was discussed. Here, not only does the life- 
guard have to consider running time in the 
different media but also the effective time 
required to climb over a phase wall with 
position-dependent height at the interface. 

A set of generalized laws of refraction 
and reflection that take into account gra- 
dients in phase shifts on optical surfaces 
enables the design of metasurfaces for steer- 
ing light waves. Consider, for example, the 
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FIGURE 9.10. Converging power of a metalens. 


metalens shown in Figure 9.10. Rays strike 
the flat lens from the left and exit the right 
side with altered phases. 

The propagation path from the surface 
point (x,y) to the focal point has the length 
d(x,y) = Vx? + y? + f? where fis the perpen- 
dicular distance from the surface to the 
focal point. Without a phase shift at the 
surface, each ray will accumulate a phase 
shift of (27r/A)d(x,y) through propagation, 
and there will be no focusing effect. On 
the other hand, an appropriately designed 
metasurface can impart to each ray a phase 
shift of the amount: 


(x,y) = Pe ERA 


As a result of these phase shifts, when 
the light rays reach the spherical surfaces 
shown by the dotted curves they will be in 
phase and will converge on the focal point 
in phase. Therefore, they will constructively 
interfere and focusing will be complete. In 


contrast, a conventional convex-convex 
lens achieves constructive interference at 
the focal point by advancing the phase of 
rays near the center to compensate for the 
longer path length for rays near the edges, 
as shown in Figure 6.3. 

The metalens illustrated here was 
designed for a particular optical wavelength; 
and chromatic aberration, i.e. wavelength- 
dependent focal length, turned out to 
be more severe than with conventional 
lenses.’ Fortunately, methods are available 
to compensate for the wavelength depend- 
ence and to produce the equivalent of an 
apochromatic triplet lens. 


9.8 Conclusion 


( “no 8 and 9 have provided an 
introduction to lens design issues, 
but they are by no means definitive. 

The interested reader can find additional 


material on websites and, of course, in 
optics texts. Chapter 10 provides a brief 


da s 
COMING TO TERMS WITH REAL CAMERA LENSES A 7 


look at fisheye lenses, Chapter 11 consid- 
ers “equivalent images” obtained with lenses 
having different focal lengths, and Chapter 
12 introduces lenses and combinations of 
lenses for close-up photography. Hopefully, 
the reader will gain some appreciation ofthe 
art, science, and technology of lens making; 
and will be motivated to pursue interest- 
ing ideas elsewhere. In spite of its age, this 
field is very active! Every day new ideas and 
experiments are reported. 
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CHAPTER 10 


Wide-Angle Lenses, Fisheye 
Lenses, and Panoramas 


In this connection it is of interest to ascertain how the external world 
appears to a fish below the surface of smooth water. The objects 
surrounding or overhanging the pond must all appear within 
the circle of light. . . . 


—ROBERT W. WOOD 


This started me to look at the world in a different way. 
Ever since, sometimes | look with telephoto eyes, 
sometimes with wide-angle eyes. 


—ALFRED EISENSTAEDT 


10.1 Introduction 


number of severe problems are 
Aceon in the design of wide- 

angle lenses with large apertures. 
First, there are the aberrations. Barrel dis- 
tortion, where vertical and horizontal lines 
tend to bow out away from the center of the 
image, is common; and corrections become 
more difficult as the focal length decreases. 
For example, full frame (FF) 35 mm cam- 
eras have an image diagonal of 43.3 mm, 
and the smallest practical focal length for 
highly corrected, large aperture, rectilinear 
lenses turns out to be about 11 mm. But 
the real problems begin as the field of view 
advances toward 180°. In this regime the 


width of the image increases without limit 
and the brightness at the edges vanishes. 
The decrease in brightness away from the 
center of the image results from the follow- 
ing effects, as illustrated in Figure 10.1: 


1. The circular lens aperture is reduced 
to a narrow ellipse when viewed from 
large angles. 

2. At larger angles a bundle of light rays 
from the lens is spread over larger 
areas (s) of the sensor. 

3. The distance (r) from the lens to the 
edges of the sensor becomes very large. 

(These effects are combined in the 
Cos* Law [Chapter 9] which states that 
the image illumination is proportional to 
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FIGURE 10.1. The geometry of light rays at different angles. 


the fourth power of the cosine of the angle 
of incidence.) 

So how can we achieve wide angles of 
view even exceeding 180°? Fisheye lenses 
provide an answer if we are willing to accept 
a special kind of distortion or rather a dif- 
ferent mapping of the 3D world onto a 2D 
plane. In fact, fisheyes can provide fields of 
view larger than 180°, and they are widely 
used in astronomy and meteorology (all-sky 
cameras), surgery (endoscopic imaging), 
surveillance (stationary CCD cameras), and 


personal security applications (peepholes). 
There is interesting science here with a long 
history. In striking contrast to rectilinear 
lenses, fisheye lenses yield circular images in 
which the hemispheric field is projected onto 
a plane. Very large distortions result from 
this projection. Figure 10.2 provides a good 
comparison of the operation ofa fisheye lens 
with a standard camera lens (rectangular). 
This illustration is based on the classic sci 
optic ball or “ox-eye lens” that was developed 
by Daniel Schwenter (1585-1636). The lens 


Incident Angle 


Sensor 


FIGURE 10.2. Image projection with a scioptic lens. 
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is mounted in a ball so that it can be rotated 
to any orientation, much like the human eye. 
The beauty of this arrangement is that the 
bundle of rays close to the optic axis form 
an accurate image of part of the hemispheric 
field at each orientation. 

With modern technology, this is suf- 
ficient because the segments of the image 
can be captured and stitched together with 
a panoramic computer program. When the 
rays are projected on the planar sensor at 
the bottom of the figure, a well-corrected 
rectilinear image is obtained similar to the 
image obtained with a pinhole camera. Of 
course, the overall brightness is greater and 
the “swing lens” permits the light flux to be 
captured more efficiently at large angles. 

In contrast to this, a fisheye lens effec- 
tively captures the image on a hemispherical 
surface like that shown by the dotted curve. 
The question then becomes how to pro- 
ject the hemisphere onto a planar surface.’ 
There are, in fact, anumber of different fish- 
eye projections in use. Figure 10.3(a) shows 


(a) 
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an orthographic projection (OP) similar to 
that used by the fisheye lens developed by 
Nikon for brightness measurements. 

The equidistant projection fisheye 
shown in Figure 10.3(b) ensures that the 
distance of an image point from the center 
is proportional to the incident angle of the 
corresponding object point. Commercial 
fisheye lenses often approximate the equi- 
distant projection. It is interesting to note 
that digital cameras are often rotated 
about their entrance pupils to acquire a set 
of images that can be used to construct a 
panorama. This is similar to imaging on 
a hemisphere with a scioptric lens. The 
“stitching” software that constructs the flat 
panorama is essentially generating part of 
an equidistant fisheye projection. Multishot 
panoramas are covered in Section 10.6. 

In comparing conventional wide-angle 
and fisheye lenses, we encounter the follow- 
ing projections where R is the distance from 
the center of the image, fis the focal length, 
and q is the incident angle: 


(b) 


Incident Angle 


R=fo 


FIGURE 10.3. (a) Orthographic projection; (b) Equidistant projection. 
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10.2 Conventional or 
Rectilinear Lenses 


ell-corrected rectilinear lenses 
provide the same projection as 
a pinhole camera. Parallel lines 


in planes perpendicular to the optic axis 
remain parallel in the image. The perspec- 
tive is distorted at the edges of the image, 
and the field of view is limited by the focal 
length of the lens and the size of the sensor. 
For example, if f= 15 mm and the sensor 
has a width of 36 mm, the horizontal field of 
view (HFOV) is 100.4° (R = f tan(¢)). 


10.3 Types of Fisheye Lenses 


he equidistant projection fisheye 

| (also known as linear scaled, angular, 

or ideal): It is convenient for angular 
measurements as in star maps (R = fp). 

The stereographic projection (SP) fish- 
eye maintains angles. Objects near the 
edges are not severely compressed, and the 
images are relatively easy to “defish” This is 
a desirable lens for interior photography. A 
stereographic fisheye lens is now available 
from Samyang; and it is sold under vari- 
ous names, e.g. Rokinon and Bower (R = 2f 
tan(g/2)). 

The orthographic projection (OP) fisheye: 
Nikon was able to implement the OP fisheye 
by using aspheric lens elements. This lens 
gives an extreme fisheye effect with objects 
at the center made even larger, and it was 
useful for measuring illumination in archi- 
tectural applications. This special purpose 
lens was expensive and not very popular (R 
= fsin(9)). 

The equisolid-angle projection fisheye: 
With this lens an element of area in the 
image is proportional to the correspond- 
ing element of solid angle in the scene. The 


mapping is similar to that obtained with a 
convex spherical mirror. One use of such 
lenses is to measure the fraction of sky cov- 
ered by clouds. This type of projection is 
found in a number of commercial full circle 
fisheye lenses for 35 mm cameras (R = 2f 


sin (/2)). 


10.4 How Fisheye Lenses 
Accomplish Their Magic 


T he term fisheye view was introduced 
into photography by Robert Wood, 
who was an optical physicist at Johns 
Hopkins University. In his text, Physical 
Optics, he invited the reader to consider the 
outside world as viewed by a fish.” Directly 
overhead, objects would be seen clearly; 
but away from the vertical direction there 
would be distortion. Also, beyond a critical 
angle, the surface of the water would be a 
mirror because of total internal reflection. 
The effect can be quantitatively understood 
by considering refraction of rays of light 
at the air/water interface as illustrated in 
Figure 10.4. 

According to Snell’s law, n,sin(0,) = 
n,sin(0,), and since n, = 1.0 for air and n, 
= 1.33 for water, the angle of incidence 0, 
is always greater than the angle of refrac- 
tion 0, (Refraction is illustrated in Figure 
6.2.) Also, we find that when 6, = 90°, the 
angle 0, of the refracted ray reaches a maxi- 
mum value of about 49°. Figure 10.4 shows 
how the refracted rays in the water, that are 
restricted to angles less than 49°, map out 
the full 180° view above water. 

The human eye is not constructed 
to be able to see this effect very well, but 
Prof. Wood recognized that it can easily 
be photographed. He, in fact, constructed 
a pinhole camera, filled it with water, and 
demonstrated that it could produce photo- 
graphs with the full 180° hemispheric view. 
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FIGURE 10.4. Fisheye view. 


Four of these photographs are reproduced 
in his text. 

In Wood's pinhole water tank camera, 
the distances from the center of the image 
are simply related to the angle of view, 
with 0° at the center and 90° at the edge, as 
illustrated in Figure 10.5. The absolute dis- 
tances, and hence the scale, depend on the 
distance from the pinhole to the film plate 
and the refractive index of the fluid in the 
tank. 

The fisheye effect can, of course, be real- 
ized with glass lenses. An obvious substitute 


Wood's Pinhole 
Camera 


Air (n,) 


for water in the pinhole camera is the 
hemispheric lens shown in Figure 10.6(a). 
However, with this lens the Petzval sum is 
large, i.e. there is a lot of field curvature. The 
field curvature can be improved by adding a 
diverging (negative) lens as in Figure 10.6(b), 
but then there are significant color aberra- 
tions. Kenro Miyamoto (1964) showed that 
the color aberrations can be corrected by 
adding a doublet, with positive and negative 
elements, before the pupil (Figure 10.6(c)).* 
Most modern fisheye lenses share features 
of this design. 


Film Plate 


FIGURE 10.5. 


Illustration of Wood’s pinhole camera. 
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FIGURE 10.6. Evolution of fisheye lenses: (a) Bond’s hemispheric lens, (b) Hills fisheye, and (c) 


modern fisheye lens by Miyamoto. 


The negative entrance lenses divert 
incident rays so that the effective aperture 
is tilted. Also, the design decreases the 
exit angles and minimizes the differences 
in path lengths for rays in different parts 
of the image. The overall effect is to can- 
cel the factors that contribute to the Cos* 
Law for rectilinear lenses and to give much 
more uniform illumination over the image. 
Basically, the uncorrected barrel distortion 
distributes the light flux into small areas at 
large distances from the center of the image 
circle to give fairly uniform illumination. 


10.5 Examples and Illustrations 

tandard fisheye lenses for 35 mm cam- 
S= give 180° fields of view and are 

available in two types. Fisheye lenses 
with focal lengths in the range 6 mm to 8 mm 
fit the complete field of view (full circle) disk 
into the narrow dimension of the image, 
while “full frame” fisheyes with focal lengths 
of 14 mm to 16 mm spread the hemispheri- 
cal image across the film or detector diagonal, 
e.g. 43.3 mm for FF sensors. However, some 
commonly used DSLR cameras have sensors 
that are approximately 15 mm x 23 mm in 


size, i.e. APS-C sensors with a crop factor of 
1.6. With the smaller sensor, an equidistant 


fisheye lens with f= 15 mm will give a HFOV 
of only about 85°. Therefore, a lot of the 
fisheye effect is lost, and we are left with a 
wide-angle image with some barrel distor- 
tion. Of course, the barrel distortion can be 
corrected with image editing software such 
as Photoshop to provide a rectilinear image. 
A similar HFOV can be obtained with a con- 
ventional lens (rectilinear) having a focal 
length of about 11 mm. It is interesting to 
note, however, that the full circle fisheye lens 
for a FF camera almost becomes a full frame 
fisheye on a 1.6 x crop factor camera. 

As noted, computer correction of barrel 
distortion in fisheye lenses is now available. 
Consider the image in Figure 10.8 obtained 
with a full frame (f = 15 mm) fisheye lens 
(Sigma EX) on 35 mm film. The yellow rec- 
tangle defines the area of the image that 
would be obtained with a crop factor of 1.6. 
Notice that each small segment of the image 
has fairly small distortion even though the 
overall image shows strong barrel distor- 
tion. The effect is similar to that of looking 
through a transparent hemisphere divided 
into many sections, freezing the scene 
observed into each segment, and then pro- 
jecting the hemisphere onto a plane. 

The apparent distortion results from 
the large angles displayed. By design, the 


WIDE-ANGLE LENSES, FISHEYE LENSES, AND PANORAMAS 


f = 15mm lens 
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FIGURE 10.7. Image circles (180% and sensor sizes. 


FIGURE 10.8. Fisheye image of a scene at the North Carolina Arboretum in Asheville. 


angular spread from corner to corner diago- 
nally is 180° and the horizontal and vertical 
fields of view are approximately 150° and 92° 
for FF sensors, respectively. Two horizontal 
lines will naturally appear to approach each 
other at the ends as required by perspec- 
tive. Now we convert the fisheye image to a 
rectilinear image by adjusting the distance 


of each image point from the center. This 
conversion was performed with Panorama 
Tools written by Helmut Dersch. This type 
of conversion can also be performed with 
the software package Fisheye-Hemi or 
the Warp tool in Photoshop. The result is 
shown in Figure 10.9 for full frame and 1.6 
crop factor cameras. 
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FIGURE 10.9. Illustration of the correction for barrel distortion in a fisheye image. 


It is, in fact, the rectilinear image that 
conflicts with reality at very wide angles by 
attempting to counteract perspective. Recall 
that parallel lines are supposed to converge 
at infinite distance. The distortions pro- 
duced by ultra-wide rectilinear lenses are 
perhaps more disturbing than the fisheye 
distortions (see Figure 10.9). However, in 
both cases we are attempting to represent 
a wide view of the 3D world with a planar 
image of limited size. It is analogous to the 
problems that map makers face in repre- 
senting the globe with flat maps. 


10.6 Panoramas 


here are two common definitions of 

panoramic photographs. The first 

requires a field of view (FOV) wider 
than that of human eyes or greater than 
about 160°. The more common definition 
requires an aspect ratio greater than 1:2. Of 
course, a “panorama” can be cropped from 
any photograph, but here I am concerned 
with a wide field of view in addition to an 
aspect ratio greater than 1:2. In the 19th and 
20th centuries a number of elaborate wide 
field panoramic cameras were produced. 
Some of these used a rotating lens to paint 


a vertical strip of light onto film that was 
in synchronized motion. These slit scan or 
strip cameras could produce a 360° image 
line by line. Digital scanning panoramic 
cameras were also marketed under the 
names Panoscan and Eyescan. In the late 
20th century, fixed lens, flatbacked pano- 
ramic cameras were produced by a number 
of companies, including Linhof and Fujifilm. 

Extremely wide panoramas, even 360° 
panoramas, based on circular arrays of 
image panels have a long history. In fact, 
Robert Barker, the painter, patented the 
concept of a circular building to display his 
panoramas in 1787 and opened a cyclorama 
building in Edinburgh, Scotland. Such dis- 
plays were extremely popular in the 19th 
century, and there was a natural evolution 
to photographic panoramas and even 360° 
movies. The emotional impact of very wide 
screen images is undeniable. 

With digital photography, capturing a 
series of overlapping images and stitching 
them into a panorama is extremely easy. 
The process is completely automatic with 
some cameras and cell phones so that the 
user has only to scan the device between 
the beginning and final points. However, the 
production of a high quality panorama still 
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requires some understanding. Suppose that 
a photographer pans a camera and collects a 
series of images. It is almost certain that the 
images will not stitch properly if there are 
any objects visible in the foreground. This 
is a consequence of motion parallax where 
nearby objects appear to move faster than 
distant objects. Fortunately, this problem 
can be eliminated by rotating the camera 
about an axis through the no-parallax point 
of the lens (see Section 9.5). The easiest way 
to accomplish error-free scanning is to use 
a panoramic tripod head with a properly 
positioned camera. 

Panoramic and virtual reality (VR) 
tripod heads are available from a num- 
ber of vendors including Nodal Ninja, 
360Precision, Sunwayfoto, and GigaPan. 
Figure 10.10 shows a camera mounted on 
a Nodal Ninja 3. This setup permits rota- 
tion of 360° around a vertical axis and +90° 
rotation about a horizontal axis through 
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the no-parallax point (NPP) of the lens. VR 
refers to an immersive environment that 
can be interacted with by means of special 
electronic equipment such as goggles, joy- 
sticks, and data gloves. VR photography is 
a more limited concept that implies interac- 
tive viewing of 360° or spherical panoramas 
in 2D or 3D. 


Rectilinear and cylindrical panoramas: The 
now discontinued Fuji GX617 camera made 
6 cm x 17 cm images on 120 or 220 film 
with lenses in the focal length range 90 mm 
to 300 mm. Beautiful, eye-catching pano- 
ramas were captured with this camera, and 
some of these are on display in galleries. 
For example, Thomas D. Mangelsen offers 
spectacular panoramas of landscapes and 
wildlife printed at 29 in. x 90 in. and even 
36 in. x 110 in.* However, the Fuji camera, 
with its flat film plane, suffers from extreme 
compromises. The lenses must project an 


FIGURE 10.10. Camera on a panoramic tripod head. 
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image circle at least 17 cm in diameter, 
and vignetting is so severe that a centered 
neutral density filter is necessary for com- 
pensation. The images are rectilinear and 
are not cylindrical panoramas. A natural 
question is whether one can replicate such 
panoramic images with a digital camera? 

The simplest way to replicate the FOV 
and the aspect ratio of the Fuji 617 camera 
is to select a focal length for the lens of the 
digital camera that gives the desired FOV 
and then to crop the image to obtain the 
6:17 aspect ratio. The FOV of the Fuji 617 
with 90, 105, 180, and 300 mm lenses can be 
obtained with a FF camera using 19, 22, 38, 
and 64 mm lenses, respectively. Similarly, 
an APS-C (1.6 factor) camera requires focal 
lengths of 12, 14, 24, and 39 mm, respec- 
tively. With this approach, many pixels are 
lost when the frames are cropped to 6:17; 
and, of course, the horizontal resolution 
is limited by the pixel count of the sensor. 
However, the images are still rectilinear. 

The way to obtain a really high-resolution 
panorama is to use a digital camera in the 
portrait orientation on a panoramic tripod 
head and to stitch several images to make 
a cylindrical panorama. With this proce- 
dure, the vertical resolution becomes 8,688 
pixels for the FF Canon 5Ds and 6,000 
pixels for the APS-C Canon 80D, and the 
horizontal pixel count is determined by 
the number of images that are stitched and 
the amount of overlap. For acquiring rapid 
and reproducible panoramas it is neces- 
sary to determine in advance the number 
of images and the step angle between them. 
I find that five images with approximately 
28% overlap work very nicely for replicating 
a 6:17 image. For this job, I used a Nodal 
Ninja 3 panoramic tripod head with the 15° 
detent ring. This inexpensive head is sat- 
isfactory for small to medium-size DSLRs 
and all mirrorless cameras. 


As an illustration I attempted to repli- 
cate the characteristics of a Fuji 617 camera 
equipped with a 105 mm lens by means of a 
Canon 7D. The Fuji 78° field of view (FOV) 
can be matched in a single shot with a 
14 mm lens in the landscape orientation on 
the 7D. The focal length of the Canon lens 
required for the five image stitch was cho- 
sen by scaling the focal length of the Fuji 
lens by the ratio of the sensor (long dimen- 
sion) to the film width, i.e. 22.3:60 or 1:2.7, 
to obtain 105 mm/2.7 = 39 mm. The results 
are illustrated in Figure 10.11 with a pair 
of panoramas obtained in Grand Tetons 
National Park. The upper one (10.11(a)) 
was made in one shot with the 14 mm lens 
and cropped. The lower one (10.11(b)) was 
acquired in five shots with a 40mm lens 
with an angular separation of 15°. 

The images in Figure 10.11 were cropped 
to an aspect ratio of a little less than 6:17; 
and the characteristic features are clear. 
The upper image is about 5,000 pixels wide 
compared with more than 12,000 pixels in 
the lower image. Also, the upper image is 
rectilinear and is close to linear perspective. 
The perspective appears to be different for 
the two images even though the camera 
position was at the same. Notice that the 
features in the center of the lower image are 
larger relative to the trees at the extremes 
than in the upper image. This center 
emphasis or “fisheye effect” results from the 
way the component images were captured 
and stitched into a cylindrical panorama. 
The cylindrical panorama can be projected 
onto a flat plane to correct for the center 
emphasis, if desired, with a program such 
as Microsoft HD View. 

The movement of the lens and sensor 
during capture is illustrated in Figure 10.12. 
For simplicity I only show three of the five 
orientations. The sensor moves on a cir- 
cular path but the images are stitched and 
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FIGURE 10.11. Vistas of the Grand Tetons. (a) single shot and (b) five portrait images stitched 
together. 
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FIGURE 10.12. Capturing images for a stitched panorama. 
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displayed in a plane. The result is similar 
to that obtained with an ideal fisheye lens 
where the horizontal separation of features 
in the image is proportional to the angu- 
lar separation of the same features during 
capture. The heavy horizontal line rep- 
resents the film plane in the Fuji camera. 
The resulting stitched image shows almost 
no vignetting and avoids the stretching 
distortion that is characteristic of ultra- 
wide-angle rectilinear lenses. 


Spherical panoramas: The panoramic head 
makes it easy to tilt a camera up or down to 
make two, three, or more rows of images. In 
fact, the GigaPan head can be programmed 
to capture hundreds or even thousands 
of images in programmed horizontal and 
vertical angular ranges with narrow field 
of view lenses. These images are stitched 
with proprietary software to create pano- 
ramas with extremely high resolution. A 
gallery of such images can be found on the 
GigaPan website. The Kolor PanoGear and 
the Roundshot VR Drive are competitive 
panning devices. 

With a 10 mm fisheye lens on an APS-C 
camera or a 15 mm fisheye lens on a FF 
camera a spherical panorama covering the 
full solid angle of 477 sr requires at least eight 
images. The procedure is as follows. First, 
the camera is tilted up from level (horizon- 
tal plane) by 30° and then rotated around 
the vertical axis to make an exposure every 
90°. For the remaining four exposures, the 
camera is rotated down to 30° below level 
and shifted by 45° around the vertical axis 
relative to the first set of images before mak- 
ing exposures every 90°. At this point we 
have eight or 24 images, when HDR (three 
shot bracketing) capture is used. 

Here I present one path for processing 
the images to produce a spherical (VR) 
panorama. First, I apply global corrections 


for lens profile, exposure, clarity, vibrance, 
sharpness, and noise reduction in 
Lightroom; and, if necessary, I merge the 
HDR sets to obtain eight tone-mapped 
images. Next the images are exported as 
TIFF files to Autopano Pro for automatic 
stitching to make the panorama. PT-Gui is 
another good choice for stitching spheri- 
cal panoramas. There is some chance that 
stitching will fail because of incomplete 
overlap of images. A more robust proce- 
dure is to acquire five images in steps of 72° 
rather than four images in each set to insure 
more complete overlap. 

As an illustration, I show in Figure 10.13 
an equirectangular projection of a spheri- 
cal panorama taken at Duke Gardens on 
an overcast spring day. This is simply a 
mapping of pixels into a rectangle so that 
the angles of rotation from —180° to +180° 
around the vertical axis are plotted on the 
horizontal axis (abscissa—x) and the corre- 
sponding angles away from the horizontal 
plane from —90° to +90° are plotted in the 
vertical direction (ordinate—y). Certainly 
no lens could make this image in a single 
shot. A hypothetical rectilinear wide-angle 
lens with decreasing focal length would give 
images that become rather abstract at fields 
of views above 120° and at ~179° degener- 
ate into lines radiating from a perspective 
point in the center of the image. In contrast 
the equirectangular projection spreads the 
north pole of the virtual sphere along the 
top border (x = —180* to +180°, y = +90°) 
and the south pole along the bottom border 
(x = —180° to + 180°, y =—90°). 

Strange as the equirectangular image is, 
it provides a convenient starting point for 
VR interactive displays and numerous other 
mappings. For example, one can select any 
direction and project a rectangular image 
with any desired field of view up to almost 
180°. Also, a mirror ball located at the NPP 


FIGURE 10.13. Equirectangular image of Duke Gardens. 


of the camera lens can be mapped with any 
desired orientation. How does one take 
advantage of these wonderful possibilities? 
Obviously one needs the right software. 
For example, the Pano2VR package per- 
mits one to generate interactive VR and a 
host of other mappings. I have also enjoyed 
Pano2VR-generated mirror ball images and 
printouts of the six faces of VR cubes that 
can be displayed in 3.5 in. photo cubes. For 
the interactive display, I chose the Adobe 
Flash output VR. VR can also be displayed 


on iPads and iPhones with PangeaVR. A 
little browsing the internet will uncover 
numerous sites with VR displays. I rec- 
ommend the Oxford tour with a link to a 
panorama tutorial by Dr. Karl Harrison.° 
The mirror ball mapped from the data in 
the equirectangular image of Duke Gardens 
with a particular choice of orientation 
is shown in Figure 10.14. Here the back- 
ground blur is a rectilinear image generated 
from the same equirectangular image with a 
field of view of 179° corresponding to a focal 


FIGURE 10.14. Mirror ball in Duke Gardens. 
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length of less than 0.15 mm for an APS-C 
sensor. The mirror ball was mapped, resized, 
and pasted onto the background. Of course, 
this is a rather unusual mirror ball because 
the photographer and camera do not appear 
in the reflection. An observer might wonder 
if this view was taken by a camera in a spy 
microbot the size of a gnat. Similar ques- 
tions arise in viewing Renaissance art where 
detailed representations of convex mirrors 
appear in paintings but seldom show the art- 
ist. For example, see The Arnolfini Portrait 
(1434) by Jan van Eyck and Saint John the 
Baptist and the Franciscan Heinrich von 
Werl (1438) by Robert Campin.° 


10.7 Conclusions 


e have shown that fisheye 
lenses are not extensions of 
wide-angle lenses, but rather 


a different optical concept. The full 180° 
hemisphere can be imaged, and with avail- 
able software the image can be converted 
to the rectilinear form if desired. Here we 
have a preview of the future of optics. If it 
is difficult to eliminate barrel distortion in 
a wide-angle lens, leave it in and correct 
the result with software. This and other in- 
camera corrections can be found in some 
modern digital cameras. But this is just the 
beginning, because the field of view can be 
extended to spherical panoramas by image 
stitching. 

At the time of this writing, cameras are 
available that produce high-resolution 3D 
spherical video. This is possible through the 
simultaneous operation of multiple wide- 
angle cameras in one enclosure or frame. 


Numerous vendors are entering the field, 
and the social media site Facebook is sup- 
porting VR images and displays. Predictions 
about the future of VR are complicated by 
the appearance of augmented reality (AR) 
or the introduction of artificial elements 
and manipulation into real-time images of 
the environment. 
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CHAPTER 14 


What Is an Equivalent Image? 


Distinctions drawn by the mind are not necessarily 
equivalent to distinctions in reality. 


—THOMAS AQUINAS 


Size matters not. Look at me. 
Judge me by my size, do you? 
Hmm? Hmm. And well you should not. 
For my ally is the Force, and a powerful ally it is. 


—YODA 


11.1 Introduction 


JE advertisements for digital cam- 
eras usually state that the lens has 
a focal length range “equivalent” to 
a certain range on a 35 mm camera. One 
“expert” tells us each lens has a focal length 
and that it is not equivalent to any other 
focal length regardless of the circumstances. 
Another expert claims that a big sensor is 
necessary to get a small depth-of-field. These 
are at best half-truths. So, what is correct? 
In earlier chapters, I explained the relevant 
background material concerning perspec- 
tive, field of view, brightness, and diffraction. 
Here I just need to bring that all together. 
This exercise provides a good review of con- 
cepts and also leads to useful conclusions. 
In particular, I show that increasing the 


F-number increases the depth-of-field, but 
can also reduce the resolution through the 
effects of diffraction. The conclusion is that 
the sensor size alone determines the maxi- 
mum useful F-number once the required 
resolution has been specified. 

It is easier to understand the important 
points by considering what is necessary to 
obtain an “equivalent” or identical image 
with different camera/sensor sizes. In order 
to obtain an equivalent image, all of the 
following properties must be the same: 


Angle of view (field of view) 
Perspective 
Depth-of-field (DoF) 


Diffraction broadening 
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Shutter speed/exposure 
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According to this definition, two equiva- 
lent images will appear to be the same in all 
respects; and the observer will not be able to 
detect any evidence of the size camera/sen- 
sor that was used. This still leaves “noise, 
pixel count, and lens quality to be consid- 
ered later. The starting point is to recognize 
that typical cameras have sensors that range 
in size from about 4 mm x 6 mm to 24 mm x 
36 mm ora factor of six in width. Let's call the 
small sensor point-and-shoot (P&S) and the 
larger sensor full frame (FF). Of course, we 
will have to enlarge the image from the P&S 
sensor a factor of six more than the image 
from the FF sensor to be able to compare 
them. 


11.2 Determinants of Image 
Appearance 


Angle-of-view: We can maintain the angle 
of view by scaling the focal length (f) as 
shown in Figure 11.1. If the lens on the FF 
camera is set for 60 mm, then the P&S must 
be set to 60 mm/6 = 10 mm. 

By scaling the focal length so that it is 
proportional to the sensor width, we have 
obtained an image that is similar in one way; 
but it may not really be equivalent when the 


Lens 


Angle of view 


other properties are considered. To under- 
stand how to make an equivalent image 
with the P&S and FF cameras, we need to 
investigate which properties can be scaled 
and which cannot. Of course, nothing about 
the real world changes when we use a dif- 
ferent sensor, and in particular there is no 
change in: 


1. the desired distance between the 
object and the lens 


2. thesize ofthe object to be photographed 
3. the wavelength of light. 


With these invariants in mind, we can 
proceed through the list of image properties. 


Perspective: This is an easy item to address 
because “perspective” depends only on the 
position of the camera lens and is unaf- 
fected by the focal length. In Chapter 4 I 
showed that the focal length has no effect 
on the perspective and only determines the 
size of the image. No matter what size cam- 
era/sensor we use, we only have to position 
the lens in the same place. The idea that we 
can change perspective by switching to a 
wide-angle lens is, therefore, incorrect. The 
perspective only changes when we move 
relative to the subject. 


Sensors FP 
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FIGURE 11.1. The horizontal angle of view for FF sensor (focal length = 60 mm) and P&S sensor 


(focal length = 10 mm). 
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FIGURE 11.2. Light rays from an object to a detector. 


Depth-of-field: Actually, only objects at 
one distance from the camera are in perfect 
focus, but the human eye has limitations, 
and objects at other distances may appear 
to be in focus in a photograph. The range 
of distances where things are in acceptable 
focus defines what we mean by depth-of- 
field (DoF). Figure 11.2 shows that a point 
source at a distance p in front of the lens is 
focused exactly at a distance q behind the 
lens, i.e. the red dotted lines come together 
at the sensor. The blue lines from another 
point more distant from the lens are focused 
at a point in front of the sensor, and when 
they reach the sensor they have diverged to 
define a circle. Rays from an object closer 
to the lens than the distance p would focus 
behind the sensor and would also define 
a circle on the sensor. These circles are 
called circles of confusion (CoC). If the CoC 
is small enough, the object is said to be in 
focus, and the distance to the object falls 
within the DoF. The subjective part of this 
calculation involves deciding on an accept- 
able CoC. After that is done, the remainder 
of the calculation is simple, albeit tedious, 
geometry. 

However, there is a caveat. The type 
of DoF computation depends on the way 
photographs will be viewed! In principle, a 
photograph should always be viewed from 
its proper perspective point. That is to say, 
the angle subtended by the photograph at 


the eye should be the same as the field of 
view of the lens used to take the photo- 
graph. Therefore, photographs taken with 
a wide-angle lens should either be printed 
very large or held close to the eye so as to 
fill much of the field of view while telephoto 
photographs should be held farther away. It 
is sometimes forgotten that perspective in a 
photograph depends only on the position of 
the lens relative to the subject (object). 

So how will the photographs be viewed? 
In fact, prints are usually viewed from a dis- 
tance of 10 in. to 15 in. regardless of the focal 
length of the lens used to make the photo- 
graph. When mounted prints are viewed, 
observers typically stand about the same 
distance from all prints. Observers gener- 
ally don’t know what focal length lens was 
used, and they simply react to the apparent 
distortions present when a wide-angle pho- 
tograph is viewed from a distance greater 
than the perspective point. Similarly, there 
is an apparent flattening effect when tele- 
photo photographs are viewed at a distance 
too close to the eye. Also, in photographic 
exhibits the audience often remains seated 
at the same distance from all prints or from 
the projection screen. Under usual view- 
ing conditions, it is appropriate to compute 
the DoF with a constant CoC in the image 
regardless of the focal length of the lens. 

So what is the CoC? This choice depends 
on how good the human eye is at resolving 
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closely spaced dots or lines. Various authors 
quote the acuity of the human eye in the 
range of 0.5 to 1.5 minutes of arc (1° = 60 
arcmin). To put this information in con- 
text, I note that 1 arcmin corresponds to the 
diameter of a dime (18 mm) viewed from 68 
yards or 1 part in 3000 of a 50” field of view. 
Of course, the contrast in photographs is 
less than that in eye test charts that involve 
black lines on white backgrounds, so we 
need to relax this requirement a bit. We 
will follow standard practice and choose 
1 part in 1500 as the defining angle. (This 
is the most conservative value suggested 
by Canon though the “Zeiss formula” uses 
1730.) What this means is that the CoC fora 
given sensor is taken to be the sensor’s diag- 
onal length (Diag) divided by 1500. For the 
FF sensor (or 35 mm film) this gives a pro- 
jection of CoC on the image plane as about 
30 micrometers while for the P&S sensor it 
is only about 5 micrometers. There is noth- 
ing sacred about this choice of CoC. It may 
be fine for an 8 in. x 10 in. print viewed from 
13 in. where the horizontal field of view is 
only 42°, but for 13 in. x 19 in. prints that 
will be viewed from the same distance the 
divisor should probably be scaled up from 
1500. Note that for 8 in. x 10 in. prints, the 
CoC on the print is only about 0.2 mm. The 
question of perceived image quality is dis- 
cussed in detail in Chapter 17. 

I start with the distance to the object (p), 
the selected focal length (f), and the CoC 
for each sensor size. This provides enough 
information for the calculation of the DoF 
for each value of the lens aperture 6 or the 
F-number (N) since the aperture equals the 
focal length divided by the N, i.e. ô = f/N. 
The necessary DoF equations, derived with 
geometry, are well-known and relatively 
simple; and the results are also well-known. 
Namely, the predicted DoF increases with- 
out limit to cover all distances as the N is 


increased without limit even though the 
actual resolution will be degraded by dif- 
fraction effects. Therefore, the maximum 
useful DoF is limited by diffraction; and it 
is necessary that we quantify diffraction 
effects before completing this calcula- 
tion. (See Appendix J.) CoC defined here is 
denoted by c in Appendix J and its projec- 
tion on the image plane is given by c’. 


Diffraction broadening: Diffraction is ubiq- 
uitous in nature, but it is usually considered 
to be a curiosity. Sometimes diffraction is so 
obvious that it cannot be ignored. We have 
all seen the rainbow of colors that appear 
when white light is diffracted from the 
surface of a CD or DVD. This occurs just 
because the rows of data pits are separated 
by distances close to the wavelength of light. 
(Actually the separation on a DVD is about 
650 nm, the wavelength of red light.) When 
we refer back to first principles, we see that 
everything having to do with the propaga- 
tion of light depends on the fact that light is 
an electromagnetic wave. Diffraction effects 
become noticeable when light waves inter- 
act with small apertures and sharp edges. 

The distribution of intensity on the sen- 
sor for a point source of light depends on 
the size and shape of the aperture. For a cir- 
cular aperture the image has a bright central 
dot encircled by faint rings as illustrated in 
Figure 5.2. This dot, the Airy disk, is well- 
known in astrophotography. The radius 
at which the intensity of the central spot 
reaches zero, i.e. the first dark ring, serves 
to define the diffraction spot size. With this 
definition, the diameter of the diffraction 
spot on the sensor is 2.44 - À - N where J is 
the wavelength of the incident light and N 
is the F-number (//6). 

The circle of confusion is the key to this 
analysis. I stated earlier that the CoC has 
been chosen to define the region where the 
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focus is good enough to be included in the 
DoF. Now [assert that high resolution in the 
DoF region also requires that the diffrac- 
tion spot be less than or equal to the CoC. 
This upper limit on the diffraction spot size 
requires that, 2.441N = CoC = Diag/1500 
and we immediately have a determination 
of the maximum useful N for any wave- 
length 1. Visible light has wavelengths in 
the range 400 um (blue) to 700 nm (red); 
and, therefore, diffraction effects are greater 
with red light. For the calculation of N, I 
have selected 1 = 555 nm, the wavelength 
that corresponds to the region of maximum 
sensitivity of the human eye. The result is 
surprisingly simple. To compute the largest 
useful N value for any camera, simply deter- 
mine the diagonal dimension of the sensor 
in millimeters and multiply that number by 
0.5. The computed values of N, CoC, and 
scaled f values for various sensor sizes are 
shown in Table 11.1. 

Here f, refers to the focal length of the 
lens on the full frame camera that serves as 
the reference. The important point is that 
the value of N computed for a given sensor 
size is the largest value that can be used with- 
out degrading the image. This calculation 
applies to an ideal lens where the perfor- 
mance is diffraction limited. Real lenses 


TABLE 11.1. 
F-numbers (N) and circles of confusion 
(CoQ) for some common sensors. 


Sensor/film Sensor F-number CoC f/f, 
Type (Diag/mm) (N) (um) 
1/2.5" 7.18 35 48 0.17 
1/1.8" 8.93 4.4 Be) K021 
2/3" 11.0 5.4 Us 1025 
4/3" 22.5 11 15 0.52 
APS-C 28.4 14 19 0.65 
35mm (FF) 43.4 21 29 1.0 
6x7 92.2 45 61 2.1 


have various aberrations and usually show 
the best resolution at a couple of N stops 
away from the largest aperture. However, as 
Nis increased (aperture is decreased), tests 
show that performance decreases primar- 
ily because of diffraction. If a photograph is 
cropped, the CoC should be adjusted to the 
new diagonal, i.e. reduced. 

Given the shooting distance (p), the 
focal length (f), CoC for each sensor size, 
and the maximum N from the diffraction 
computation (Table 11.1), we have every- 
thing required to determine the DoF. For 
this calculation, I have used a DoF equation 
that is appropriate for a constant CoC (or 
c’) in the image. This equation is derived in 
standard optics texts (see Appendix J), and 
I have chosen an approximate form that 
is accurate when the distance p is much 
greater than the focal length fand the DoF 
does not extend to infinity. 


_2.pP«N-CoC_2.p*.CoC 


DoF P fo 


What we find is that the DoF is the same 
for all sensor sizes because of scaling of the 
focal length and F-number. Also, the lens 
aperture 6 remains the same regardless of 
sensor size. For example, suppose that the 
image of a 2 ft. object completely fills the 
diagonal dimension of the sensor when 
photographed from 5 ft. For a FF camera 
this requires a focal length of about 100 
mm. The optimum F-number is 21, the 
aperture is 5 mm, and the DoF turns out to 
be about 10 in. In contrast to this, the P&S 
camera with the smaller sensor achieves an 
equivalent image with a 17 mm focal length 
and an F-number of 3.5. 


Shutter speed/exposure: This final prop- 
erty is very interesting. Suppose that the 
FF camera in the last example required 
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1/30 sec at an ISO sensitivity of 800 for 
an adequate exposure. The P&S camera 
with F-number 3.5 admits (21/3.5)? = 36 
times more light intensity and can achieve 
the same exposure with a shutter speed of 
1/1000 sec. This calculation depends on the 
idea that the brightness quadruples when 
the F-number decreases by a factor of two. 
On the other hand, the equivalent photo- 
graph requires that we use the same shutter 
speed to make sure that motional blur is 
the same. Therefore, in order to obtain the 
proper exposure, we must decrease the ISO 
sensitivity of the P&S camera by a factor 
of 36 to obtain an ISO number of 22. As it 
happens, most cameras cannot go this low 
in sensitivity; but maybe we can select 50. 
This, of course, demonstrates that small 
sensors do not have to perform well at high 
ISO settings in order to obtain an equiva- 
lent image with maximum useful DoF. 


11.3 Comments and Conclusions 


different sizes can be used to make iden- 

tical images if the appropriate ranges of 
focal lengths, apertures, shutter speeds, 
and ISO sensitivities are available. This also 
assumes that lenses have sufficient resolv- 
ing power and that the pixel density is high 
enough not to degrade resolution. 

We can make equivalent images with 
any sensor size by: 


fe demonstrated that sensors with 


a) Scaling the focal length to maintain the 
field of view (FOV). 


b) Holding the aperture (5) constant to 
maintain the depth-of-field (DoF). 


c) Holding the shutter speed constant to 
maintain the motion blur or lack of it. 


d) Scaling the ISO sensitivity to maintain 
the brightness. 


In following this plan we make the focal 
length proportional to the width of the sen- 
sor and the ISO setting proportional to the 
square of the sensor width. 

The concept of an equivalent image is not 
very controversial, and is generally consid- 
ered to be a useful idea. Also, in creating the 
equivalent image we change the focal length 
to what some people call an equivalent focal 
length. This idea is also used in referring to 
a focal length as equivalent to focal length 
on a FF camera. Of course, the focal length 
may not be equivalent for the DoF, and 
the idea of an equivalent focal length has 
been criticized. However, the idea is useful 
because the field of view is so important to 
most people. This is especially true for those 
buying small point-and-shoot cameras and 
cell phone cameras where the DoF tends to 
be very large in all cases. 

Also, when the focal length is scaled 
while the aperture is held constant, the 
F-number, N, changes so that it becomes 
inversely proportional to the crop factor. 
This increases the exposure of the image and 
requires the ISO sensitivity to be reduced 
as described above to maintain the bright- 
ness of the image. The scaled F-number has 
been called the equivalent F-number. This 
use of “equivalent” is more difficult to justify 
because the F-number is defined to specify 
the light intensity on the sensor regard- 
less of the sensor size. The term equivalent 
F-number is not really needed and can be 
confusing. 

There are a few more points that are 
worth emphasizing: 


1. The sensor size alone determines the 
maximum useful F-number (N); and, 
in fact, the maximum F-number for 
high-resolution photography is given 
by 0.5 x (sensor diagonal in mm) 
at least for prints 11 in. x 14 in. and 
smaller. 
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2. The smallest sensor we considered 


(denoted 1/2.5 in.) achieves maximum 
useful DoF at an F-number of N = 3.5 
while the full frame 35 mm sensor 
requires N = 21 for a factor of 36 dif- 
ference in transmitted light intensity. 
A P&S sensor that could give low noise 
at ISO 400 or 800 would appear to have 
a real advantage over FF sensors. 


. If maximizing the DoF is not the aim, 
larger sensors clearly win because of 
their ISO sensitivity advantage. A fast 
lens (N = 1.2) with a full frame detector 
is impossible to match with the small 
sensor. Probably N = 1 is the maximum 
aperture we can expect with a small 
sensor, and few companies at present 
even offer N = 1.5. The take home les- 
son is that small sensors should be 
coupled with large aperture lenses, i.e. 
small N values. Also, small sensors that 
support larger ISO sensitivities should 
be sought. The vendors are showing 
some interest in higher sensitivities, 
but larger apertures are in conflict with 
their drive to reduce camera sizes. 


. Image Quality (IQ) was not consid- 
ered in defining the equivalent image 
(see Chapter 17). Subjective IQ covers 
both signal-to-noise ratio (S/N) ratio 
and the perceived resolution. S/N 
usually depends on sensor size and, 
in general, is higher for larger sensors. 
This is related to the fact that the total 
number of photons captured is higher 
for a larger sensor. The image resolu- 
tion depends both on the lens and the 
pixel count. Each pixel typically con- 


converter program to combine infor- 
mation from neighboring pixels to 
assign three color values (R, G, and B) 
to each pixel site. As a rule of thumb, 
we should at least double the size of a 
pixel to approximate its effect on reso- 
lution. Therefore, when the number 
of RGB pixels on the diagonal divided 
by two approaches 1500, the number 
used to define the CoC, and pixel size 
begins to influence resolution. A 24 
megapixel sensor with a 2:3 aspect 
ratio has 6000 pixels on the largest 
side, and the safety margin is not very 
large. Fortunately, the IQ for images 
from digital cameras in 2016 is usually 
quite good and will increase with new 
technology for years to come. 


. The fundamental limitation on DoF is 


imposed by diffraction effects. Does 
technology offer any hope for cir- 
cumventing the diffraction barrier? 
Probably not, because in photogra- 
phy we collect light far removed from 
the source (far field radiation). In this 
region, information about fine details 
in the object being photographed is 
contained in the spread of angles cap- 
tured by the camera lens. The larger 
the lens aperture, the wider the spread 
of angles that can be collected. Smaller 
apertures lose high-frequency informa- 
tion in the image and that is the source 
of diffraction broadening. This effect is 
independent of the quality of the lens. 


Further Reading 


tains one sensor element, so nothing Fowles, G. R. (1975). Introduction to Modern 
smaller than one pixel can be resolved. Optics, 2nd Ed. New York: Dover Publ. 
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in most sensors each pixel only records Rochester, NY: Case-Hoyt Corp. 
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cHAPTER 4 D 


How to Get Very 
High Magnification 


Macro photography refers to close-up photography; the classical 
definition is that the image projected on the “film plane” 
(i.e film or a digital sensor) is close to the same size 
as the subject. 


—WIKIPEDIA 


Life is a tragedy when seen close-up, 
but a comedy in longshot. 
—CHARLIE CHAPLIN 


12.1 Introduction 


picture frame are often called close-ups, 

but that is not the subject here. This 
chapter is concerned with real close-ups 
and “macro” photography. This distinction 
is made clear by the concept of magnifica- 
tion, m, which just means the ratio of the 
size (height) of the image of an object to 
the actual size (height) of the object (see 
Chapter 7): 


[re of objects that fill up most of the 


m = (image size)/(object size) (12.1) 

Standard camera lenses permit maxi- 
mum magnifications from about 0.1 to 0.3; 
and those at the high end of this range often 


have a “macro” label displayed on the lens 
barrel, on a push button, or in an accom- 
panying menu. True macro lenses permit 
magnifications at least to 1.0. This is also 
called lifesize, 1X, or 1:1. Similarly, m = 0.5 
might be denoted as % X or 1:2. 

The magnification concept is the key to 
understanding the essential features of both 
telescopic and close-up photography. In the 
photography of birds and animals the mag- 
nification is usually quite small (m < 0.1). 
A lion may fill up the entire frame, but the 
image height will only be 24 mm or less for 
DSLRs and smaller cameras. In this regime, 
the magnification is proportional to the 
focal length of the lens and also inversely 
proportional to the distance from the lens 
to the object. This explains the need for very 


92 oy CHAPTER 12 


“long” lenses to photograph distant animals 
and birds. In the absence of a long lens, the 
photographer has no recourse but to get 
closer to the subject. 

At the opposite extreme, e.g. plant and 
insect close-ups, the magnification is often 
0.25 or greater. Strange as it may seem, 
in this region the magnification can be 
increased by decreasing the focal length. 
This is, in fact, the effect of “close-up” sup- 
plemental lenses. The magnification can 
also be increased by increasing the distance 
from the lens to the sensor, i.e. adding 
extension. These may sound like arbitrary 
rules, but, in fact, they follow directly from 
the simple conjugate equation. 

Here, again, science unifies apparently 
disparate sets of observations, and it is 
worthwhile to spend a few minutes to see 
how everything fits together in conformity 
with the laws of optics. This will require a 
few equations. If equations are not helpful 
for you, please skip to the next section. For 
simplicity, I will begin with a single thin lens 
(Figure 12.1), but the results can easily be 
generalized to compound lenses of any com- 
plexity. (Note that the right triangle formed 
by ends of the object and the center of the 
lens is similar to the right triangle formed 
by the ends of the image and the center of 
the lens. Therefore, (image height)/(object 
height) = q/p.) 


Object 


m = q/p 


As shown in Chapter 7, the working 
distance, p, and the image distance, q, are 
related to the focal length, f; by the conju- 
gate equation: 


tytn (12.2) 
p d 


In the telescopic limit, the object is 
located so far to the left that p is much 
greater than the focal length f. In this limit, 
light rays from the object form an image in 
the plane where q = f and the magnifica- 
tion is close to zero. Now suppose we move 
the object closer and closer to the lens to 
increase the magnification. When p = 2f, it 
is easy to show with the help of Equation 
(12.2) that p = g and m = 1. This benchmark 
arrangement is shown in Figure 12.2. 

Now suppose we need a magnification 
greater than 1X. One way to accomplish this 
is to move the object closer to the lens so 
that p < 2f. Of course, the image will not be 
in focus unless q can be increased to satisfy 
Equation (12.2). As p approaches the value 
f, the rays on the right-hand side become 
parallel and the image plane is infinitely 
far from the lens (on the right). The con- 
clusion is that the entire “macro” range (m 
= 1 to m = ~) is confined between p = 2f 
and p = f. Of course, when p is less than f, 
the image is virtual, and the lens functions 


FIGURE 12.1. Magnification with a single thin lens. 
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FIGURE 12.2. The arrangement for 1X magnification. 


as a magnifying glass. (Virtual means that 
the image appears to be where it does not 
actually exist, as with reflections in a mir- 
ror. A real image exists where the rays from 
a point on an object are brought together in 
another point.) 

This discussion may leave the false 
impression that focus is always obtained by 
moving the lens relative to the sensor. This 
type of focusing, moving the entire lens unit, 
is known as unit focusing. The important 
parameter in unit focusing is the extension, 
or distance added between the lens and the 
sensor. Focus may also be obtained by the 
movement of elements inside a compound 
lens. For example, the front element may be 
moved relative to the other lens elements to 
produce a change in the focal length. Modern 
compound lenses may, in fact, use both exten- 
sion and focal length change to achieve focus. 

Useful expressions for the magnification 
can be obtained from Equation (12.2) and 
the definition m = q/p. For example, it is 
easy to show that 


= 1 


12.3 
F (12.3) 


m 


Therefore, in the macro range we 
can increase the magnification by either 
increasing q (the total extension) or by 
reducing the focal length f In the other limit 
where p is large and q/f approaches 1, the 
magnification vanishes. 


Another useful expression for m 
(obtained by multiplying Equation (12.2) 
by p and rearranging) is 


f 


ee 12.4 
oA) vom 


m 


In the telescopic limit where p is much 
greater than f we find that m = f/p. This 
result justifies our comment that at great 
distances the magnification is proportional 
to the focal length. 

Equation (12.2) appears to have all the 
answers, but there is a catch. Practical cam- 
era lenses are compound lenses. The object 
distance p is measured to the first nodal 
point and the image distance q is measured 
from the second nodal point as described in 
Chapter 9. Therefore, the distance from the 
object to the sensor is usually not equal to 
the sum of p and q. Furthermore, as men- 
tioned above, the internal movement of lens 
elements may change the focal length as well 
as the distance of the lens elements from the 
sensor. The bottom line is that the simple 
equations shown above are only useful for 
compound lenses when the nodal points 
are close together or when the locations 
of the nodal points are known. 

Another factor that must be considered 
is the increase in the effective F-number, 
Ny» relative to the value of N, reported by 
the camera, that occurs when the magni- 
fication is increased. This, of course, can 
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greatly increase exposure times and exac- 
erbate the effects of diffraction. It turns 
out that N,,, is related to the magnification 
m by the equation N, = N(1 + m), except 
when magnification results from the use 
of a supplemental close-up lens. However, 
there is a caveat. Lenses are characterized 
by an entrance pupil and an exit pupil. For 
a “symmetrical lens” the entrance and exit 
pupils are equal, but in general they can 
be quite different. Typically, the entrance 
pupil is larger than the exit pupil for tel- 
ephoto lenses, while for wide-angle lenses 
the exit pupil is larger than the entrance 
pupil. This can be verified by simply looking 
into a lens and estimating the apparent size 
of the aperture stop from the front and the 
rear. The pupil ratio is defined as p, = (exit 
pupil)/entrance pupil), and a more accurate 
equation for the effective F-number is: 


Ng=N- (14%) (12.5) 


So where does this leave us? The simple 
conjugate equation is not very useful for 
macro photography, the positions of the 
nodal points for the lens are in general not 


available, and the entrance and exit pupil 
diameters can only be estimated. (Actually, 
these numbers have been reported for 
a few prime lenses.) The situation is not 
ideal, but with a modern digital camera we 
can measure all the quantities needed for 
macrophotography. 

Consider the standard set of close-up 
accessories: 


1. Macro lens or close-focusing tele- 
zoom lens 


2. Supplemental lenses (1 to 50 diopter) 
3. Extension tubes or bellows 
4. Teleconverter (1.4X or 2X). 


With a prime lens and any combination 
of supplemental lenses and extensions in 
place, the following quantities can be meas- 
ured (or estimated) at close focus and far 
focus (ce): 


1. Working distance: w (distance from 
object to front of lens combination). 
2. Distance from object to sensor plane: D. 


3. Magnification: m (from measurement 
of image height on sensor). 


FIGURE 12.3. Illustration of the working distance w and the distance D to the sensor for a cam- 
era with accessories: (1) supplementary close-up lens, (2) macro lens, (3) extension tubes, and 


(4) tele-extender. 
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4. The effective F-number: N F (from the 
automatic aperture setting at fixed 
shutter speed). 


5. Resolution as a function of No" 


The accessory lenses and distances are 
shown in Figure 12.3. 

In the next few sections I will cover the 
various combinations and arrangements for 
increasing the magnification and point out 
their advantages and limitations. 


12.2 Supplemental Lenses 


robably the easiest and least expensive 
p way to obtain close-up or even macro 

capability with a standard lens is to add 
a supplementary lens. The simplest of these 
is a one- or two-element screw-in lens that 
attaches just like a filter; though, of course, 
they are not filters. These close-up lenses are 
sometimes called diopters or plus diopters 
because their strength is rated in diopters. 
(Recall that the optical strength is propor- 
tional to the inverse focal length or d = 1/f 
and the dimensionless strength in diopters is 
obtained when the focal length is measured 
in meters.) If the focal length is measured in 
millimeters, as is usually the case, the opti- 
cal strength is d = 1000 mm/f= 1000/(f/mm) 
diopters. For example, 100 mm lens has a 
strength of d = 1000/100 = 10 diopters. 

The magnification expected from a 
camera lens (prime lens) with an attached 
diopter can be estimated with the sim- 
ple lens equations. The basic idea is that 
the strength for the combination of two 
closely spaced lenses is just the sum of their 
strengths in diopters. This result can be 
expressed either in terms of diopters as d = 
d, +d or focal lengths: 


Prime Diopter 


1 1 1 
= + 
catia vom Sie 


(12.6) 
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Suppose that the prime lens is described 
by foime = 100 mm and a +2 diopter close- 
up lens is added. The effect is to decrease 
the combined focal length and hence to 
increase the optical strength. In this case 
the strength becomes 10 diopters + 2 diop- 
ters and the combined focal length is 83 
mm. The magnification can easily be esti- 
mated when the prime lens is set at the 
far-focus (+) position so that q =f,,, This, 
of course, means that all parallel rays enter- 
ing the prime lens will be brought to focus 
in the plane a distance f, from the lens. 
Also, we know that rays from a point on 
an object at a distance p = foiopter from the 
close-up lens will be parallel when they exit 
the plus diopter. As illustrated in Figure 
12.4, the combination of a close-up lens 
with a prime lens always gives a working 
distance (w) equal to the focal length and a 
magnification is 


me Jom (127) 


Diopter 


Equation (12.7) is useful for standard 
lenses with limited focusing ranges. For 
the 100 mm lens with a +2 diopter supple- 
mentary lens, we obtain m = 0.2. As a test, 
I attached a +2 diopter lens to a zoom lens 
and set the focal length to 100 mm. I found 
that at far focus the working distance was 
approximately 50 cm and the magnification 
was m = 0.20. When the lens was focused 
as closely as possible, the working distance 
was reduced to 16 cm, and I measured m = 
0.27. As a more extreme example, I attached 
the +2 diopter to a 105 mm macro lens that 
is capable of 1X without attachments. Here 
again, at far focus the working distance was 
50 cm and m = 0.2; however, at close focus 
the working distance is only 8.5 cm and m 
= 1.24. With the macro lens, it would be 
crazy not to use the close focus capabilities, 
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Prime lens 
(Camera lens) 


FIGURE 12.4. The operation of a close-up lens when the prime lens is focused at infinity. 


and Equation (12.7) is not very helpful. Still 
the close-up lens increases the maximum 
magnification a bit by reducing the effec- 
tive focal length. This can be useful when 
the amount of extension is limited. 
Supplementary lenses have impor- 
tant advantages. They are small and easy 
to use, they have little effect on the effec- 
tive F-number,' and their contribution to 
the magnification increases with the focal 
length of the prime lens. The main drawback 
is their effect on optical quality. Inexpensive 


50mm lens 
reversed 


Macro coupling ring 
49mm to 58mm 


sets of diopters typically contain single ele- 
ment lenses and it is necessary to stop down 
the prime lens to maintain acceptable reso- 
lution. I have obtained good results with a 
set of HOYA coated diopters (+1, +2, and 
+4). However, in general photographers are 
well advised to use two-element (achro- 
matic) close-up lenses whenever possible. 
These are now available from a number of 
manufacturers. I know from experience that 
high-strength, single element diopters, e.g. 
d = 10, can severely limit resolution. 


Telephoto lens 


FIGURE 12.5. The combination of an Olympus 50 mm lens, a macro coupler, and a Canon 70-300 
mm lens that permits magnifications of 4X and greater. 
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I have saved the best part for last. It is 
possible to obtain very high quality supple- 
mentary lenses with high optical strengths 
up to at least +20 diopters. Furthermore, 
these lenses are widely available at reason- 
able prices; and many people may already 
own one or more of them. I am, of course, 
referring to fairly short focal length, large 
aperture camera lenses that can be reversed 
and used as supplementary lenses. A 50 mm, 
J/1.8, Canon lens makes a fine +20 diopter 
close-up lens and can easily be mounted 
on a variety of prime lenses by means of 
“macro coupler” reversing rings. I have 
also used lightweight Olympus OM system 
50 mm, //1.8, and 100 mm, //2.8, lenses as 
+20 and +10 diopter supplementary lenses. 
Lenses like this are available on the used lens 
market at very reasonable prices. The com- 
bination of a telephoto lens with a reversed 
50 mm lens is shown in Figure 12.5. 


12.3 Extension 


nother common method for mak- 
ing any lens focus “closer” is to add 
some extension between the lens 
and the sensor plane. A convenient way to 
accomplish this for DSLRs is to use a set of 
extension tubes. For example, the set sold 
under the name Kenko includes 12 mm, 
20 mm, and 36 mm tubes that can be used 
separately or stacked together to give 68 mm 
of extension. These tubes contain no glass 
elements but do provide electrical connec- 
tion between the prime lens and the camera 
body for automatic focusing and exposure 
control. It is also possible to obtain exten- 
sion bellows with a focusing rail for some 
cameras. These systems with (automatic) 
electrical connections are very expensive and 
are rarely used by amateur photographers. 
So how much good does added exten- 
sion do? As an example, consider the Canon 


50 mm, f/1.8 II lens. At close focus the dis- 
tance from the object to the sensor plane 
(D) is approximately 44 cm, the working 
distance (w) is 34 cm, and the measured 
magnification (mm) is only 0.15. Close focus- 
ing with this lens only adds a small amount 
of extension and q remains approximately 
equal to the focal length. Now I add an addi- 
tional 68 mm of extension with the entire 
set of stacked tubes. With this addition, the 
measured magnifications for close and far 
focus become 1.46 and 1.31, respectively. 
This is true macro capability, but the combi- 
nation is not very flexible, and the working 
distances range is only from 4.8 cm to 5.3 
cm! Furthermore, the effective F-numbers 
are approximately 2.5 times larger than the 
values of N reported by the camera. 

The following, often quoted equation, 
relates the magnification to the added exten- 
sion: m = (added extension) /(focal length). 
This equation is, of course, just Equation 
(12.3) with q = f + added extension. It gives 
some guidance for standard prime lenses, 
but in real-world macrophotography, where 
one usually works with zoom lenses and/or 
macro lenses having considerable built in 
extension, the equation is almost useless. 
The Sigma 105 mm macro lens proves the 
point. With 68 mm of added extension the 
magnification ranges from 0.66 to 1.89 in 
going from far to close focus because of the 
considerable extension in the lens mount. 
The only practical way to characterize a 
combination of lens and extension tubes is 
to measure its performance. The macro lens 
with extension tubes may be practical when 
magnifications greater than 1X are required, 
but here again there is considerable loss of 
light. I estimate the effective F-numbers are 
two to four times the reported values. 

In general, extension becomes cum- 
bersome as the focal length increases. 
However, the increase in magnification 
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FIGURE 12.6. A rose photographed in the late afternoon without flash (f/11, 1/30 sec). Background 


blur was increased by computer processing. 


with an extension tube may be more than 
expected because the prime lens is likely to 
achieve close focus by decreasing the focal 
length. Also, a small amount of extension 
can be useful with short focal length (wide- 
angle) lenses to obtain striking images of 
extended objects such as almost flat flow- 
ers. Of course, fisheye and ultra wide-angle 
lenses often focus very close without any 
extension. For example, the photograph in 
Figure 12.6 was taken with a 15 mm fisheye 
lens (APS-C size sensor). 

The conclusion is that extension tubes 
can be useful and are well worth owning. 
They are relatively inexpensive and offer 
increased magnification without decreasing 
optical quality. Also, they can be a lifesaver 
when your favorite telephoto lens does not 
focus close enough. In this situation, an 
added extension tube may well do the trick. 


12.4 Teleconverters 
(Tele-Extenders) for Cameras 
with Interchangeable Lenses 


eleconverters are lens accessories 
T™ are usually mounted between 
a lens and a camera body in order 
to increase the effective focal length of the 
lens. Teleconverters, sometimes called tele- 
extenders, are negative lens elements (or 
groups) that are not capable of forming an 
image on their own. Peter Barlow (1776- 
1862) recognized that the introduction of 
a diverging lens element would multiply 
the focal length while maintaining approxi- 
mately the same overall length of the lens. 
The basic idea is the same as that encoun- 
tered in the design of telephoto lenses and 
was even noted by Galileo in about 1609. 
As illustrated in Figure 12.7, the focal length 
of a simple or compound prime lens can be 
stretched by introducing a negative (diverg- 
ing) lens after it. The combination produces 
a simple telephoto lens with the focal length 
f, and the telephoto ratio is defined as L/f. 
When simple lenses with paraxial rays 
are involved (Chapter 7), the focal length of 
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FIGURE 12.7. The effect of a teleconverter or Barlow lens on rays from a prime lens. 


the closely spaced combination is related to 
focal lengths ofthe separate components by 
Equation (7.6). 

In the real world, with long telephoto 
lenses and rather thick teleconverters, 
simple equations are not very useful. The 
important point is that the teleconverter 
provides a magnification increase by the 
factor m = s,/s,. This means that a 100 mm 
lens can act like a 200 mm lens at least at 
far focus, but at any focal distance the 
magnification is increased by the same fac- 
tor. Furthermore, this factor increases the 
magnification already achieved by macro 
lenses or extension tubes. Therefore, con- 
verters act as multipliers of magnification. 
Teleconverters also preserve the working 
distance as well as the full focusing range 
of the prime lens out to infinity. So, what is 
the down side? Everything in photography 
involves compromise. With teleconvert- 
ers, we lose light intensity. Since the focal 
length increases while the aperture remains 
the same, the F-number must increase. 
The 1.4X teleconverter costs one stop and 


the 2X unit costs two full stops. Besides 
this, the introduction of additional glass 
always reduces the quality of the image. 
The decrease in quality may be small with 
1.4X, especially if the lens is stopped down; 
but with 2X and higher magnifications the 
reduction can be noticeable. 

Even with some reduction in image 
quality, the use of a 2X teleconverter may 
be worthwhile. The Juza photo site has 
reported excellent macrophotography with 
a 180 mm macro lens combined with a 2X 
teleconverter.? This combination provided 
increased working distance with impres- 
sive resolution at small apertures when 
stopped down to f/11. Of course, diffrac- 
tion is the limiting factor. The effective 
F-number is at least twice as large with the 
2X teleconverter, and the rule of thumb is 
that diffraction becomes significant when 
the F-number exceeds one half of the sen- 
sor diagonal expressed in mm, i.e. about 
f/14 for an APS-C (1.6 crop factor) type 
sensor. 
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12.5 Afocal Teleconverters 
for Fixed Lens Cameras 


ompact cameras and less expen- 
sive cameras usually do not have 
interchangeable lenses. Therefore, 
telephoto and wide-angle accessories for 
these cameras must be attached in front of 
the camera lens. This is analogous to using a 
fixed lens camera to take photos through the 
eyepiece of a telescope, thus replacing the 
human eye with a camera. For this to work 
with any camera lens, an afocal arrangement 
(illustrated in Figure 12.8) is required. The 
magnification in this arrangement is given 
by m = f/f, = fgl fiamera When the camera is 
focused on infinity. For closer distances the 
actual magnification measured from the 
image size on the sensor depends somewhat 
on the focal length of the camera lens. The 
same principles can be used to make a front 
mounted wide-angle converter. 
Afocal teleconversion lenses can be quite 
satisfactory for point-and-shoot cameras 
and camcorders. They also work well on 


lenses designed for DSLRs, but tend to be 
bulky, and they offer no advantage over the 
rear mounted teleconverters. In fact, there is 
a serious problem for close-up photography. 
In contrast to rear mounted teleconverters, 
the front mounted units greatly increase 
the minimum focusing distance. So there 
might be an advantage when photograph- 
ing a bird at 10 ft., but at close distances 
the maximum magnification is actually 
decreased. In addition, afocal teleconverters 
usually lack threads for mounting polarizers 
and filters, and they may put unacceptable 
stress on lens barrels. In particular, they are 
not recommended for lenses with rotating 
front elements. 


12.6 Combinations of 
Lenses and Extenders for Real 
World Macrophotography 


have shown that magnification can be 
increased by changing the focal length 
and/or changing the amount of exten- 
sion. Your best course of action, of course, 


camera 
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FIGURE 12.8. Front mounted teleconverter. 
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depends on the application and the equip- 
ment at hand. There are pros and cons for 
the various options. Here is a summary 
that repeats some previous advice and adds 
some information about useful combina- 
tions of techniques. I am assuming that 
extreme magnification is not required and 
that a maximum of 1X suffices with occa- 
sional excursions to 3X or 4X. 


Macro lenses: It is convenient to shoot 
everything from birds to small insects with 
a macro lens on an interchangeable lens 
camera. Large apertures (f/2.8) and long 
focal lengths (100 mm, 180 mm, etc.) that 
permit good working distances and reason- 
able exposures, when there is good light, 
are readily available. The working distance 
can be increased when needed by adding 
a 1.4X or 2X teleconverter lens at the cost 
of reduced light transmission and perhaps 
image quality. When higher shutter speeds 
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are necessary, supplementary light sources 
may be required (see Section 12.8). It should 
be noted that Canon also offers a special 
purpose manual focus macro lens (MP-E 
65 mm f/2.8) for magnifications from 1X to 
5X. Experienced photographers can obtain 
remarkable results with this lens, but it is 
not for everyone. I recommend experimen- 
tation with magnifications less than 5X 
before considering the MP-E. Figure 12.9 
shows photographs taken with a 105 mm 
macro lens in good light. 


Supplementary lenses: A diopter increases 
magnification by permitting closer focus- 
ing without much change in extension. The 
diopter makes up for limited extension by 
decreasing the effective focal length of the 
camera lens. Because the increase in mag- 
nification is offset by the decrease in focal 
length, the effective F-number is unchanged 
and there is little loss of light. The cons are: 


FIGURE 12.9. Photographs taken with a 105 mm macro lens: (a) an argiope illuminated by a 
bright sky and reflector with a gray card in the background (f/6.7, 1/250 sec); (b) a praying mantis 


illuminated by flash (f1, 1/15 sec). 
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1. loss of image quality because of the 
added glass 


2. loss of working distance 


3. limited focusing range. 


The image quality problem suggests it is 
not a good idea to stack many supplemen- 
tary lenses together or to use supplementary 
lenses in combination with rear mounted 
teleconverters. However, a single diopter of 
medium strength, e.g. +2, in combination 
with a zoom lens can be quite useful when 
a light mobile setup is needed for butterfly 
photography. 


Extension tubes and teleconverters: Given 
enough extension one can obtain large 
magnification, but this is often not very 
practical. Large extensions are bulky and 
very sensitive to vibration. Also, the effec- 
tive F-numbers are large and long exposures 
are required. It is usually a better idea to use 
a moderate amount of extension, i.e. less 
than the focal length of the lens, to enhance 
the magnification already obtained with 
a macro lens or to reduce the minimum 
focusing distance of a telephoto lens. It is 
also possible to obtain high quality macro 
images by combining the use of supplemen- 
tary lenses and extension tubes. The effect 
of the added diopter is to amplify the effect 
of the extension. 

Another popular technique is to use 
both extension tubes and a teleconverter. 
The first question is about the best arrange- 
ment of the components. We know that a 
rear mounted teleconverter increases the 
effective focal length, but it also multiplies 
whatever magnification has been obtained 
by the combination in front of it. The added 
extension produces magnification equal to 
the added extension divided by the focal 
length of the lens at least when the prime 
lens is focused at infinity. For example, a 105 


mm macro lens at far focus has the magni- 
fication m = 0, but with a 68 mm extension 
the magnification becomes m = 68/105 = 
0.65 (measured value 0.66). When a 1.4X tel- 
econverter is added without any extension, 
the magnification remains zero at far focus. 
Now we use both multiplier and extension. 
First attach the teleconverter to the lens and 
then the extension. The result is to increase 
the focal length to 1.4 x 105 mm = 147 mm 
followed by the effect of extension, and we 
find a magnification of m = 68/147 = 0.46 
(measured value 0.47). Now we reverse the 
order and first attach the extension to obtain 
0.64. The teleconverter is then attached, and 
the result is m = 1.4 x 0.64 = 0.90 (measured 
value 0.93). It is obvious, that at far focus, the 
best plan is to mount the extension first fol- 
lowed by multiplication with a teleconverter. 
These calculations are easy and accurate but 
may be useless because no one does macro- 
photography with their macro lens focused 
at infinity! 

So, what is the real story about the stack- 
ing order? The conclusions reached above 
have some validity for normal lenses with 
limited focusing ranges, but for macro 
lenses and lenses with a macro mode, the 
stacking order is far less important. This 
is to be expected because macro lenses 
have a lot of extension built in at close 
focus. Consider again the 105 mm macro 
lens that gives 1X magnification at close 
focus. When I mounted 68 mm extension 
on the lens followed by 1.4X, I obtained m 
= 2.82 at a working distance of w = 7 cm. 
The reverse, 1.4X followed by 68 mm, gave 
m = 2.71 and w = 9.5 cm. It makes sense 
always to attach the teleconverter last (clos- 
est to the camera) even though the order 
might not make much difference at close 
focus. An extensive set of measurements 
for the Sigma 105 mm macro lens with 
various combinations of extension tubes, 
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diopters, and a teleconverter are presented 
in Appendix E. An immediate conclusion is 
that an extension tube in combination with 
a teleconverter gives a versatile, high mag- 
nification setup. 


12.7 Special Problems Resulting 
from High Magnification 


he accessories and their combi- 
Sk: described in Section 12.6 

all permit large magnifications to 
be obtained. Magnification is, of course, 
required for macrophotography, but that 
is only the beginning. Most of the arrange- 
ments require that we work with large 
F-numbers. Additional requirements may 
include: 


1. Photographing moving objects that 
require high shutter speeds. 

2. Optimizing F-number for depth-of-field 
and resolution at high magnifications. 


3. Enhancing the depth-of-field within 
the restrictions imposed by diffraction. 


A number of decisions are required here, 
and the restrictions become severe at the 
highest magnifications. As always photog- 
raphy involves compromises. Section 12.8 
provides some suggestions for dealing with 
the most common problems. 


12.8 Lighting for Close-Up 
Nature Photography with 
High Shutter Speeds 


or butterflies and other large insects, 
Fi. necessary magnification is not very 
high and in bright sunlight there may 
not be a problem. I recommend the following: 


1. a long focal length macro lens 
2. a zoom lens with a diopter, or 


3. a close focusing telephoto lens with a 
short extension tube if necessary to 
reduce the minimum focusing distance. 


This can be very effective for butterflies 
and small birds, as illustrated in Figure 
12.10. If the subjects are much smaller, e.g. 


FIGURE 12.10. Tropical butterfly photographed with a 70-300 mm (at 240 mm) IS lens hand-held 
(f/6.7, 1/90 sec). Supplemental flash enhanced the illumination. 
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FIGURE 12.11. Aphids and intruder taken with a 105 mm macro lens with extension (f/11, 1/125 


sec). 


ants and aphids, the magnification and the 
effective F-number may become large so 
that high shutter speeds will require supple- 
mental light sources. The aphids in Figure 
12.11 were illuminated with bright sunlight 
and a flash. The only way to stay mobile is 
to use flash attachments mounted on the 


camera lens or on brackets. A number of 
vendors sell macro ringlight flash units that 
mount on the camera lens. 

These units are good for clinical work, 
photographing coins, etc., but they can give 
flat, shadowless lighting that does appear to 
be natural. The best bet for portable macro 


FIGURE 12.12. Flash with off-camera extension cord and a mounting bracket. 
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lighting in the garden is to use brackets for 
mounting one or perhaps two flash units 
close to the lens. Kirk, Manfrotto, and oth- 
ers offer a variety of brackets. 

A Manofrotto telescoping camera/flash 
bracket is shown in Figure 12.12. Another 
possibility is to mount slave flash units near 
a target area so that they can be triggered 
either by a camera mounted transmitter or 
the light from a flash unit. 

Several inexpensive slave units can be 
used to illuminate a subject as well as the 
background. However, since some flash 
units emit preflashes for focusing or red eye 
reduction, the slave units must be able to 
select the appropriate flash for the exposure. 


12.9 The Optimum F-Number for 
Macrophotography 


ecall that the depth-of-field becomes 
P very shallow at high magnifications. 
For close-up and macrophotography 

the depth-of-field can be expressed as: 


A iy 


DOF = (12.8) 


Here CoC is again the circle of confu- 
sion, N gis the effective F-number, and m 
is the magnification. The calculation of the 
depth-of-field proceeds by selecting an 
acceptable value of CoC to define resolution 
that is “good enough.’ The choice of CoC is 
subjective, but I previously suggested that a 
reasonable value would be the sensor diago- 
nal (Diag) divided by 1500. This divisor is 
probably larger than necessary for small 
prints and too small when large prints are 
planned. The maximum resolution possi- 
ble and the minimum depth-of-field would 
require that CoC be set equal to approxi- 
mately two times the pixel width for the 
sensor in use. For example, an APS-C type 


sensor with 10.1 megapixels has Diag = 28 
mm and a pixelpitch of 5.7 um. The rule 
Diag/1500 gives 19 um, while two times the 
pixel dimension is 11 um. 

Next a value of N, is chosen to obtain 
the desired DoF. However, as discussed in 
Chapter 11, the diameter of a diffraction 
broadened spot in the image is approxi- 
mately 2.44 - A - N p and this quantity 
controls the resolution when N,, is large. 
If the maximum DoF is desired, then the 
maximum N, value consistent with the 
CoC must be determined. This is easily 
accomplished by setting the diameter ofthe 


diffraction spot equal to the CoC. 
2.44 -À -N,, = CoC = Diag/Div (12.9) 


The value of N to set in the camera is 
obtained by combining Equations (12.5) 
and (12.9): 


(Diag/Div) 
= 2 AR 12.10 
2.441(1+m/p) i 
The substitutions, Div = 1500, A = 555 
nm, and p, = 1, produce a useful expression 
for N. 


7 Diag 
— (2 mm)(1+m) iul 


Equation (12.11) is similar to the rule of 
thumb presented in Chapter 11, except for 
the inclusion of the magnification. 

As an example, consider a setup with 
Diag = 28 mm and p, = 1. With the resolu- 
tion specified by Div = 1500, I find that the 
maximum allowed value of N is 7, i.e. f/7. 
The corresponding DoF is about 0.5 mm. 
This may seem restrictive, but consider what 
happens as the magnification increases. 
When m = 2, the maximum N value is 4.7 
and DoF = 0.13 mm; and when m = 4, the 
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maximum N is 2.8 and DoF = 0.03 mm. If 
one is willing to sacrifice resolution for DoF, 
then a smaller value of Div can be chosen, 
but the depth-of-field at large magnifica- 
tions is still paper thin. For this reason, it 
is very hard to get acceptable photographs 
with magnification above 4X. Patience, good 
technique, and a lot of practice are required. 


12.10 Expanding the Depth-of-Field 


o how can the DoF be expanded? The 

laws of physics cannot be violated, and 

there is no way to capture an image 
that simultaneously offers high magnifica- 
tion (>1X), high resolution, and large DoF. 
There are, however, ways to combine a 
series of photographs obtained with differ- 
ent focal points to produce an image that 
appears to have a much larger depth-of-field 
than any one of the component images. This 
is a much more difficult task than stitch- 
ing together images to increase the field of 
view or even combining images obtained 
with different exposures to generate an 
image with a higher dynamic range (HDR). 
When the focal point is shifted, objects that 
are out of focus may shift, become larger, 


and become fuzzy. Combining a set of such 
images requires judicious cropping and 
pasting (or erasing), and the attempt may 
fail to produce an acceptable image. 

Fortunately for photographers, very 
clever programmers have devoted a lot of 
time to automating the job of compositing 
images with differing focus points. The pro- 
grams HeliconFocus, Zerene Stacker, and 
CombineZP (freeware) are results of this 
work. Also, focus stacking is available in 
Photoshop CC. These programs have been 
reviewed and compared online. A photog- 
rapher must produce a set of images with 
different points of focus and store them 
as a “stack.” The stack is then specified as 
input for the compositing program. The 
program rescales and rotates the images as 
needed and aligns them. The sharp parts of 
each image are selected and then blended 
to make the final composite. An example is 
shown in Figure 12.13. 

The production of a satisfactory com- 
posite at high magnification may require 
30 or 40 images, and depending on the 
amount of overlap of image components 
at different distances from the lens, the 
automated result may not be satisfactory. 


FIGURE 12.13. A fly trapped on a sundew leaf. The left-hand shot was taken at f/5.6, 1/8 sec; the 
image on the right was constructed by CombineZM software from eight photographs. 
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FIGURE 12.14. A focusing rail that is convenient for capturing images with different focal points. 


Sometimes manual manipulation of the 
composite images or selection of images 
to remove from the stack is necessary. This 
requires experience and trial and error. It 
is, of course, essential to capture enough 
images that the regions of good focus over- 
lap. I have found that a focusing rail with 
micrometer adjustment is much more sat- 
isfactory for stepping through the focusing 
range than trying to change the focus of the 
lens. An arrangement for macrophotogra- 
phy with a Manfrotto focusing rail is shown 
in Figure 12.14. This tedious procedure has 
been automated in the StackShot Macro 
Rail Package by Cognisys. Another option 
is offered by the CamRanger wireless DSLR 
control system. 


12.11 Conclusions 


hotography atits best gives us new ways 
to see the world. Macrophotography 
permits us to see things that are almost 
invisible to the naked eye and details that 
escape our attention. Without high magni- 
fication, I would not know the beauty of a 


single deadly drop on a sundew leaf (shown 
in Figure 12.13) or be able to see the inter- 
action of aphids with other insects. Also, 
the details of butterflies and spiders would 
elude me, and the tiny flowers of the sundew 
plant would remain unappreciated. All of 
these examples are from nature, and nature 
seems to provide the best hunting ground 
for the macrophotographer. Through natu- 
ral selection the biosphere creates a vast 
array of structures ranging from nanopar- 
ticles to whales by construction with one 
atom at a time. And in the micro and macro 
(hundreds of microns) range, nature is vastly 
more prolific and interesting than the crea- 
tions of technology up to this time. So, for 
the time being, there is not much interest 
in the macro photography of inanimate 
objects outside the science laboratory. 
Thus, coins and paper money only serve 
to demonstrate magnification and to test 
resolution. 

Whether or not the macrophotogra- 
pher feels limited to the world of nature, 
macrophotography is now accessible to 
almost anyone with enough interest. The 
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least expensive digital cameras offer macro 
modes that are surprisingly good though 
with very short working distances. For 
those able to afford more expensive equip- 
ment, the capabilities are truly impressive. 
Mirrorless cameras and most DSLRs pro- 
vide “live view” that permits the live image 
on the sensor to be magnified tenfold to aid 
in achieving focus. The image can also be 
displayed in real time on a computer screen, 
and this opens the possibility of video at the 
micro scale. Macro lenses are getting better 
and better, and computer processing meth- 
ods such as image combination open new 
possibilities every year. I think it is a great 
time to investigate this area of photography. 
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Notes 


1. Plus diopters have little effect on the 
F-number. It is easy to understand what 
is happening here. As shown in Figure 
12.4, neither the distance to the sen- 
sor nor the aperture are changed by the 
added diopter. Therefore, the effective 
F-number, which is the distance (q) to 
the sensor divided by the aperture (6), is 
unchanged. Another way to understand 
the effect of the added diopter is the fol- 
lowing. The diopter effectively reduces 
from foame tO Je, as described in 
Equation (12.6), and this means that the 
defined F-number is reduced to Nf- po! 
foime) At the same time the magnifica- 
tion becomes M = fo... foi) Which is 
equal to (fone feconn) — 1. The final step 
is to use the expression for effective 
F-number to obtain N = N AES 1 PAR 
(1 + m) = N. This more lengthy deriva- 
tion shows that the intensity increase 
expected from the reduced F-number 
is cancelled exactly by the increase in 
magnification. Furthermore, the magni- 
fication results from the fact that the lens 
to sensor distance (q) exceeds fo ppo 

2. Juza, E. A., Macro and Flora Photography: 
http://www.juzaphoto.com/article. 
php?l=en&article=15 


CHAPTER 13 


Do We Need Filters Anymore? 


Filters allow added control for the photographer of the images 
being produced. Sometimes they are used to make only subtle 
changes to images; other times the image would simply not 


be possible without them. 


—FILTER (PHOTOGRAPHY) WIKIPEDIA 


Digital cameras are excused from most of the color 
conversion filters, since you dial these in as 
white balance settings. 
—HTTP://WWW.KENROCKWELL.COM/TECH/FILTERS.HTM 


13.1 Introduction 


o you have a digital camera, and it 
Suse great photographs. Could you 

do any better by using filters? As the 
lawyers say, it depends. There are two types 
of filters you might consider. First and most 
important is the polarizing filter. This can 
give control over reflections from glass, 
water, other shiny surfaces, and light scat- 
tered from the sky. Then there are neutral 
density (ND) filters. These special purpose 
filters are less important, but can help you 
deal with scenes where there are extreme 
variations in brightness such as a dark fore- 
ground with a bright sky. They can also 
help emphasize motion by permitting long 


exposures. The remaining type of filter is 
the color filter. The effects of these filters 
can be simulated by digital processing and 
are no longer necessary except for UV and 
IR photography where visible light must be 
blocked. This chapter explains in detail how 
ND, color, and polarizing filters work. 1 will 
start with definitions and go rather far into 
the theory. There is also a bit of history you 
might find interesting.* 

Photographic filters are transparent plates 
that are designed to modify the properties of 
light before it reaches the film or sensor of a 
camera. Filters usually are mounted in front 
of the camera lens, though some large lenses 
permit rear mounting. ‘The filters of interest 
here fall in the following categories: 
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1. Neutral density (ND) filters that atten- 
uate light at all wavelengths. 

2. Color filters that modify the spectral 
distribution of the transmitted light 
by attenuating the intensity at certain 
wavelengths. 

3. Polarization filters that transmit light 
polarized in a certain direction. 


I will discuss these filters primar- 
ily in terms of their usefulness for digital 
photography. 


13.2 Absorption Filters 


Neutral density filters: High quality optical 
filters are coated to minimize reflections, 
and they are uniform throughout so that not 
much light is scattered. The light that enters 
the filter is either absorbed or transmitted. 
Both ND and color filters usually oper- 
ate by absorbing light, the exception being 
interference filters (dichroic) that selec- 
tively reflect various frequencies. The law 
of absorption has been rediscovered several 
times and is associated with three different 


Transmittance 


T=1/, 
Optical Density 
OD =- log,,(T) 


T=10% 


names. Logically it could be called the Law 
of Bouguer, Lambert, and Beer, even though 
Pierre Bouguer (1698—1758) was the first to 
demonstrate that the attenuation of light by 
a series of thin layers follows a geometric 
progression.’ As illustrated in Figure 13.1, 
there is an exponential decrease in intensity 
which means that the total transmittance, 
can be obtained by multiplying the trans- 
mittance of the layers. 

If the incident intensity is J, and each 
layer transmits 50%, then after two layers 
the transmittance is 0.5 x 0.5 = 0.25; and 
after three layers it becomes 0.5 x 0.5 x 0.5 = 
0.125. On the other hand, the optical density 
(OD) increases in a linear fashion, propor- 
tional to the thickness of the filter and the 
concentration of absorbing centers, so that 
OD = 0.3 corresponds to a transmittance 
of 50% and OD = 0.6 gives a transmittance 
of 25%. For example, a neutral density filter 
might be specified as (ND 0.9, 3, 8x), indi- 
cating an OD of 0.9 or 3 stops reduction in 
light. In other words, the light will be atten- 
uated by a factor of 8. Additional examples 
are listed in Table 13.1. 


FIGURE 13.1. The absorption process occurring in the dotted circle on the left is described on 
the right side, where the transmittance, T, is plotted versus the Optical Density, OD. 
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TABLE 13.1. 
ND Filters 
Optical Transmittance Transmittance Filter 
Density (T) Stops factor 
(OD) 
0.3 0.5 1 2 
0.6 0.25 2 4 
0.9 0.125 8 8 
2.0 0.01 6.6 100 
3.0 0.001 10 1000 
4.0 0.0001 13.3 10,000 


Photographers are usually looking for 
fast lenses, sensitive films, and sensors 
that permit good exposures at high shutter 
speeds, but there are exceptions. At times 
one may wish to use a large aperture to 
control the depth-of-field in bright light. 
This is usually possible by reducing the ISO 
sensitivity to 100 or perhaps 50 and increas- 
ing the shutter speed, but an ND filter may 
still be required. A more extreme situation 
exists when very slow shutter speeds are 


required to blur motion. This is common 
practice in photographing moving water. A 
more or less standard view of a waterfall can 
be obtained with a 1/50 sec exposure, but 
below 1/25 sec the blur becomes noticeable 
and interesting effects can be obtained at a 
half-second and longer. Slow shutter speeds 
are especially advised for waterfalls with 
weak flow. 

It may be possible to accommodate 
slower shutter speeds by reducing the ISO 
setting and by increasing the F-number with 
the accompanying increase in depth-of-field. 
However, for very long shutter speeds there 
is no alternative to an ND filter. Nature pho- 
tographers can often benefit from the use of 
a 10-stop filter to increase the exposure time 
by a factor of 1000. For example, Figure 13.2 
shows a 20-sec exposure of a quiet time at 
Thunder Hole in Acadia National Park. The 
smooth water dramatically changes the 
impact of the scene. 

The graduated filter is a special case of 
the neutral density filter. Typically one end, 


FIGURE 13.2. A 20-sec exposure of Thunder Hole (Acadia National Park). 
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FIGURE 13.3. Graduated filters illustrating hard gradient, soft gradient, radial, and radial inverse. 


about half, of these filters is completely 
clear (OD = 0), and the other end is tinted 
with a soft or hard transition between the 
regions. Distributions with different shapes 
as well as densities are also available, as 
illustrated in Figure 13.3. For example, the 
tint may decrease radially from the center. 
The purpose is to compensate for excessive 
luminance differences in a scene that exceed 
the dynamic range of the film or sensor. 
These filters can be very effective in con- 
trolling bright sky with a dark landscape. 
However, graduated filters only work with 
simple tonal geometries. Complicated 
scenes such as those with bright light 
through windows or scattered reflections 
from water competing with sky may not lend 
themselves to a simple gradient tonal filter. 


Fortunately, filters with irregular edges 
can be simulated quite easily when process- 
ing an image in Photoshop or Lightroom. A 
graduated gradient can be introduced in any 
direction, and the gradient can be applied to 
exposure, saturation, sharpening, and other 
effects. When shaping is desired to avoid 
applying the gradient in certain areas of the 
image, the brush tool in Lightroom can be 
applied to erase the effects of the gradient. 
This effect can also be obtained with the 
gradient tool and layer masks in Photoshop. 

Selective brightness effects can also be 
obtained by means of simple tonal adjust- 
ments in image processing. Figure 13.4 is 
an example of simple tonal geometry. This 
image was processed with simple high- 
lights/shadows adjustment in Photoshop. 


FIGURE 13.4. Wide-angle sunset image with computer-adjusted highlights and shadows. 
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However, when dark areas in a digital 
image are brightened by image process- 
ing, it is likely that undesirable noise will be 
enhanced. 

A better way to achieve high dynamic 
range is to acquire a set of images with expo- 
sures ranging from —3 stops to +3 stops or 
some other appropriate range that permits 
the brightest as well as the darkest parts of 
the scene to be properly exposed. The set 
of images can then be combined with the 
appropriate software to obtain a single image 
with 32 bits of brightness information. The 
tonal range can then be adjusted before the 
image is reconverted to 16-bit for projec- 
tion or printing. This kind of processing is 
called high dynamic range (HDR) and is cur- 
rently available in Lightroom, Photoshop, 
Photomatix Pro, Nik HDR Efex Pro, HDR 
Expose, and a few other software packages; 
but a word of caution is in order. The image 
“enhancements” should not be extreme, and 
a lot of trial and error is required to achieve 
consistently good results. 

The example in Figure 13.5 shows the 
interior of a church with bright windows 
ona sunlit day. The image on left is a single 
exposure, and the one on the right combines 
three images obtained with the exposure 


compensations (—2, 0, +2). The merging of 
images was accomplished with Photomatix. 
The term HDR actually refers to the image 
capture step when the exposure range 
completely covers the dynamic range of 
the scene. The resulting 32-bit HDR image 
with the full dynamic range usually cannot 
be displayed in full on monitors, prints, or 
projectors, and so the merged image must 
be tone mapped to achieve a lower dynamic 
range that can be displayed. 

Tone mapping amounts to redistrib- 
uting the tones from an HDR image to 
a lower dynamic range image that can be 
completely displayed or printed on avail- 
able media. What we actually end up with 
is a low dynamic range image (LDR). It is 
in the post-processing steps that one has 
the latitude to create either garish or pleas- 
ing images. The landscape artist facing a 
scene in nature often has the same problem. 
The brightness range of paint on canvas is 
limited to about 1 to 100, while a scene in 
nature may offer a range of 1 to 10,000 or 
more. There are many ways to accomplish 
the mapping and none of them is perfect. 

Another processing method that has 
gained some favor is fusion. This does not 
involve an intermediate HDR image, but 


FIGURE 13.5. St. Conan’s Kirk on Loch Awe: (left) single exposure, (right) three exposures with 


HDR tone mapping. 
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FIGURE 13.6. Color transmission by absorption filters: Red (A), Yellow (G), and ND. 


rather blends or averages the bracketed 
images to obtain detail in the darkest and 
brightest areas. The results are usually 
less extreme than tone mapping and may 
be very pleasing. Fusion based processing 
in an option in some HDR programs, and 
LR/Enfuse is a stand-alone program that is 
available as donationware. 


Color filters: As noted, color filters have been 
very important in photography for a century 
but have been largely replaced by white bal- 
ance adjustments in digital photography 


Kodachrome slide 


Simulated yellow filter 


and simulated filters in the conversion of 
color images to monochrome. Figure 13.6 
illustrates what color filters do. 

The filters are, of course, named for the 
color of light that is transmitted in addition 
to detailed numbering in the Wratten sys- 
tem. Figure 13.7 shows the conversion of a 
color image to monochrome with enhanced 
transmission of yellow and red light by 
means of the “Channel Mixer” feature of 
Photoshop software. Of course, subtle tonal 
changes in color images can be obtained 
through the use of “warming” filters, “sunset 


a 


Simulated red filter 


FIGURE 13.7. The conversion of a Kodachrome slide to monochrome with simulated yellow and 


red filters. 
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filters’, etc. but, in principle, any such tonal 
changes can be simulated through software 
manipulations. There are more possibilities 
when a digital image is converted to mon- 
ochrome because the color information 
is retained and can be used to selectively 
brighten or darken on the basis of hue. 


13.3 Polarization Filters 


olarization selection continues to be 

extremely important in photogra- 

phy, and its effects cannot be easily 
simulated in software. Understanding this 
phenomenon requires a consideration of 
the interaction of light with matter. For our 
purposes, light can be considered to be an 
electromagnetic wave with electric (E) and 
magnetic fields (H) perpendicular to the 
direction of propagation. The direction of 
the electric field defines the direction of 
polarization. The ray shown in Figure 13.8 
is plane polarized (transverse magnetic, or 
TM), has the wavelength A, and is propagat- 
ing to the right with the speed c. The basic 
idea is that molecules tend to absorb light 
more strongly when their long axis is par- 
allel to the direction of polarization. The 
effect is pronounced for molecules aligned 
in crystals, and the orientation depend- 
ent absorption is known as dichroism. In 


I 


general, some kind of alignment of atoms 
or molecules is required for dichroism. 
Efficient and inexpensive linear polarizers 
are now available because of the insight 
of E. H. Land, the founder of the Polaroid 
Corporation.’ 

Unpolarized light consists of a collec- 
tion of rays like that shown in Figure 13.8, 
but with random directions for their elec- 
tric fields. In the following figures, I show 
only the electric field amplitudes since elec- 
tric fields dominate absorption processes. 
Notice that in Figure 13.9 the electric fields 
initially lie in planes distributed around the 
z-axis. When the bundle of rays encoun- 
ters the polarizer, the y-components of 
the electric fields are absorbed, and the x- 
components are transmitted. The result is a 
filtering process that passes an x-polarized 
beam. Figure 13.9 implies that the absorb- 
ing units are aligned in the y-direction. A 
perfect polarizer would pass 50% of the 
intensity; but, of course, there are reflections 
and other losses that limit the transmission. 

Linear polarizers have been used success- 
fully in photography for several decades, but 
with modern cameras problems sometimes 
arise from the interaction of polarized light 
with mirrors in the autofocus system and in 
the viewfinder. The efficiency of reflection 
depends on the state of polarization, and 


mai 


FIGURE 13.8. Polarized light with the wavelength X. 
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FIGURE 13.10. The interaction of an unpolarized beam of light with a circular polarizer. 


reliable operation may require unpolarized 
light. When it is desirable to take advantage 
of polarized light in a scene, there is a way 
to analyze for polarization and still insure 
that only scrambled polarization enters the 
camera. For this purpose, one uses a circu- 
lar polarizer in place of the linear polarizer. 
The operation of a circular polarizer is 
illustrated in Figure 13.10. A linear polar- 
izer selects the direction of polarization and 
then the polarized beam passes through a 
quarter-wave plate. The result is circularly 
polarized light in which the electric field 
vector rotates around the z-axis. The linear 
polarizer and the quarter-wave plate are 
actually glued together and act as a single 


filter. Within the camera, the circular polar- 
ized light acts like unpolarized light, and the 
problems vanish. 

So what is a quarter-wave plate? Here 
we encounter doubly refracting (birefrin- 
gent) crystals such as calcite. These crystals 
are anisotropic, and the index of refraction 
depends on the orientation of the polari- 
zation of the incident light. It turns out to 
be possible to cut some birefringent crys- 
tals into slabs (plates) so that the directions 
of maximum (slow axis) and minimum 
refractive index (fast axis) are perpendicu- 
lar to each other and parallel to the surface 
of the slab. The quarter-wave plate works 
like this. The birefringent plate is oriented 
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so that the direction of polarization of the 
incident light lies between the fast and 
slow axes at to each. As shown in Figure 
13.11 the electric field E of the light wave 
can be resolved into the components E, 
and E, that lie in the directions of the fast 
ands slow axes of the crystalline plate. Just 
as 2+2 is equivalent to 4 for integers, the 
sum of the component vectors E, and E, is 
equivalent to E. As I discussed in Chapter 6, 
the apparent speed of light in a transparent 
medium with refractive index n is just c/n 
where c is the speed of light in a vacuum, 
and the effective path length for light is the 
actual distance traveled multiplied by the 
refractive index. The quarter-wave plate is 
constructed by adjusting the thickness d so 
that the difference in path lengths for the 
components along the fast axis (refractive 
index n) and the slow axis (refractive index 
n,) is one quarter wavelength or 90° phase 
shift. This is best expressed with the equa- 
tion d (n, — n,) = M4, which can be solved 
for the thickness if the refractive indices 
and the wavelength are known. The result 
is that two waves are produced, in perpen- 
dicular planes, that have equal amplitudes 
and are shifted in phase by a quarter wave- 
length, and it can be shown that the sum of 
these two waves rotates around the z-axis as 
illustrated in Figure 13.10.* 


It has been suggested that polarizing 
filters can be stacked to “increase” the polar- 
izing effect, i.e. to narrow the distribution 
of light that is passed by the combination 
(from cos?0 to cos*0). This procedure will 
certainly fail for two circular polarization 
filters because the output of the first filter 
would be scrambled so that the second filter 
would not be able to improve the selection. 
Similarly, rotating one of the filters relative 
to the other would not change the fraction 
of light transmitted by the pair. The situ- 
ation is different for a linear polarization 
filter before a circular polarization filter 
where distribution is indeed narrowed (to 
cos*@). Also, rotation of one of the filters 
would attenuate the light completely when 
the filters were crossed. This combination 
could serve as a variable neutral density 
filter. Two linear polarizers would behave 
the same way, except the output from the 
second filter would not be scrambled. 


13.4 Polarization in Nature 


hotographers are interested in polar- 
izers because they want to be able to 
control naturally polarized light as 
it enters a camera. Polarized light usually 
results from reflection or scattering. I first 
consider scattered light. This is not a simple 


FIGURE 13.11. Resolution of polarized light into components. 
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topic, but a simplified model captures all the 
important features; and we only need two 
ideas to explain the phenomenon: 


1. The electric field of light exerts force 
on charged particles and, in particu- 
lar, causes electrons in atoms and 
molecules to accelerate parallel to 
the direction of the field. These elec- 
trons oscillate at the frequency of the 
incident light wave. 


According to Maxwell’s electromag- 
netic theory, accelerating electrons 
emit radiation. This is well-known 
from radio theory where current 
oscillates in antennas and broadcasts 
a signal to listeners. (Recall that accel- 
eration is the rate of change of velocity. 
This naturally happens in an oscilla- 
tion where the velocity periodically 
changes direction.) 


These ideas apply to cases where there 
is little or no absorption, and the scattered 
light has essentially the same frequency as 
the incident light. Figure 13.12 shows the 
angular distribution of radiation for an 
oscillating electron. The distance from the 
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origin to the red curve at a particular angle 
indicates the amount of light intensity in 
that direction. The important points are: (i) 
the direction of polarization is established 
by the direction of oscillation, and (ii) there 
is no intensity directly above or below the 
oscillator. 

Consider, for example, a ray of sunlight 
in the sky, well above the horizon. It is easy 
to verify with a polarizing filter that the 
light exhibits very little polarization when 
one looks toward the sun or away from the 
sun. Maximum polarization is at 90 from 
the direction of the sunbeam, and the sky 
becomes quite dark as the filter is rotated. 
Figure 13.13 illustrates the situation. When 
looking toward the sun, we receive radiation 
from electrons oscillating in all directions 
that are perpendicular to the beam direc- 
tion, but when the observer is in the plane of 
polarization all electron oscillation appears 
to be in one direction. When viewed from 
90°, i.e. the observer is in the plane of polari- 
zation, the polarization may reach 75% to 
85%. The polarization does not reach 100% 
because the oscillating electrons reside in 
molecules that are not perfect scatterers, i.e. 
the molecules are not perfectly spherical in 
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FIGURE 13.12. The radiation pattern for a dipole radiator. 
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FIGURE 13.13. The transverse electric fields associated with unpolarized light from the sun. 


shape and have a non-vanishing size; and, 
perhaps more importantly, some of the 
light is scattered by more than one electron 
before reaching the observer. Multiple scat- 
tering is the reason that light scattered near 
the horizon is not polarized. 

Reflections are a major source of polar- 
ized light in nature. In the situations 
discussed here, light is reflected from 
a very smooth, nonconducting surface. 
(Surfaces that conduct electricity strongly 
absorb light, and the treatment presented 
here does not apply.) In such cases, the 
fraction of the intensity reflected depends 
on both the angle of incidence and the 
direction of polarization of the light beam. 
Figure 13.14(a) illustrates the reflection and 
refraction of polarized light with transverse 
magnetic fields (TM), which means that the 
electric fields are in the plane of the figure 
as indicated by the arrows (perpendicular 
to the scattering surface). In Figure 13.14(b) 
the incident beam is polarized with trans- 
verse electric fields (TE), meaning that the 


electric fields are perpendicular to the figure 
surface as illustrated by the x’s (parallel to 
the scattering surface). The indices of refrac- 
tion above and below the reflecting surface 
are denoted by n, and n, respectively. 
From previous discussions (Chapter 
4) we know that the angle of incidence is 
equal to the angle of reflection 0, = 0,), and 
the angles of incidence and refraction are 
related by Snell's law (n, sin0, = n, sinó ). 
The physical description of reflection and 
refraction is as follows. The electric field of 
the incident light beam penetrates the glass/ 
water interface and induces electric cur- 
rents (oscillating electrons). The currents 
produce both the reflected and the refracted 
beams and cancel the beam in the direction 
of the undeviated dotted line. Notice that 
the electric fields of the three beams (I, A, B) 
are all parallel in (b), and we expect that the 
currents will produce reflected light regard- 
less of the angle of incidence. In contrast to 
this, the electric fields of the three beams 
(I, A, B) are not parallel in (a). Since the 
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(b) 


FIGURE 13.14. Reflection and refraction of light: (a) transverse magnetic field and (b) transverse 


electric field. 


oscillating electrons move in the direction 
of the electric field and do not radiate in the 
direction of their motion (Figure 13.12), it 
is clear in the TM case (part a) that no light 
will be reflected when the refracted (A) and 
reflected (B) beams are perpendicular. 

The remarkable conclusion is that, when 
unpolarized light is reflected from a smooth 
surface at one particular angle, the compo- 
nents polarized perpendicular to the surface 
are not reflected at all. At this angle, known 
as the Brewster angle, the reflected light 
is polarized parallel to the surface, i.e. out 
of the plane of the figure. Brewster’s angle 
can be determined by a simple calculation. 
[When paths A and B in Figure 13.14(a) are 
perpendicular, the angle of the refracted 
beam is given by 0, = 90° — @,. This expres- 
sion can be combined with Snell’s law and 
the identity sin (90° — 8.) = cos 6, to obtain 
n,/n, = sin 0,/ cos 0, = tan 0,.] The conclu- 
sion is that 0, = tan”(n,/n,). For example, a 
light beam in air that reflects from the sur- 
face of water (n = 1.33) at the angle tan”! 
1.33 = 53.06°, or 36.94° measured from the 
reflecting surface, is totally polarized. 

Of course, there is more to the problem 
than computing the Brewster angle. The 


percentage of reflected light from the TM 
and TE beams can be computed at all angles 
of incidence. This can be done by making 
use of simple geometric arguments or by 
applying Maxwell’s equations with appro- 
priate boundary conditions to describe the 
electric and magnetic fields. The Maxwell 
equation approach, while more demand- 
ing, does permit reflection from absorbing 
material to be treated. Since these calcula- 
tions would lead us too far afield, I simply 
present the results and provide references 
for the complete derivations. The fraction of 
light reflected for TM and TE arrangements 
is described by Equation (13.1). 


R iai r aa] 

TE \sin(@,+6,)|’ ™ |tan(0,+0,)| (13.1) 
These quantities are plotted in Figure 13.15 
for the special case with n, = 1 and n, = 
1.33. 

Polarization (TM/TE) at the Brewster 
angle (53.06°) is clearly shown, but it is 
also evident that polarization is strong 
from at least 0, = 40° to 60°. The results are 


slightly different for glass where n, = 1.5. In 
summary, at glancing incidence (0 = 90°) 
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FIGURE 13.15. Percent reflectance for light at the air/water interface. 


reflectance is close to 100% for both TE and 
TM components, and the reflected beam is 
not polarized. At normal incidence (0 = 0) 
about 4% is reflected with no polarization. 
Between glancing and normal angles there 
is some polarization, and it becomes com- 
plete at the Brewster angle. 


Polarization in a rainbow: Rainbows are 
favorite subjects for photographers, but few 
realize that the rainbow light is polarized. 
First, recall that rainbows are circles, and 
their center is always the shadow of the head 
of the observer, i.e. the anti-solar point. It 
turns out that the light from the rainbow 
is polarized tangent to the colored bands 
or perpendicular to a radial line from the 
center (see the yellow arrows in the photo 
in Figure 13.16). This happens because 
sunlight is reflected inside a water droplet 
and the droplet lies in the plane contain- 
ing the observer’s head and the shadow of 
the observer's head. Furthermore, the angle 
is close to the Brewster angle for internal 
reflection. The geometry of scattering is 


shown in Figure 13.16. Colors result from 
the dispersion in the index of refraction 
of water that ranges from 1.344 at 400 nm 
(blue) to 1.3309 at 700 nm (red), but reflec- 
tion (inside water droplets) is responsible 
for the polarization. 

A study of the possible angles for internal 
reflection, when there is only one reflection 
inside a droplet, shows that the maximum 
angle (from the anti-solar point) is 40.5° 
for 400 nm light and 42.4° for 700 nm light. 
This gives the primary rainbow, where the 
internal angle of reflection is about 40° and 
the Brewster angle is close to 37°. The sec- 
ondary rainbow results from two internal 
reflections, and I find that minimum angle 
for 400 nm light is 50.34° and for 700 nm 
is 53.74”. In principle, the secondary rain- 
bow has about 43% of the brightness of the 
primary, but it is almost invisible in Figure 
13.16. The internal reflection angle for 
the secondary rainbow is about 45°—not 
quite as close to the Brewster angle as with 
the single reflection. The consequence of the 
limiting reflection angles is that the colors 
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FIGURE 13.16. Refraction and reflection of light beams in rainbows: (a) primary and (b) secondary. 


are reversed in the secondary rainbow 
going outward from red to blue, and that 
no light is reflected between roughly 42° 
and 52” by either single or double reflection. 
This gives rise to the dark band between the 
primary and secondary rainbows that was 
described by Alexander of Aphrodisias in 
200 CE. and bears his name. Rainbows also 
result from three, four, and higher num- 
bers of reflections in droplets. The tertiary 
rainbow appears as a ring around the sun 
at 137.5” and is extremely difficult to see. 
Higher order rainbows up to order 200 
have been seen in the laboratory with laser 
light. Additional discussion and illustra- 
tions can be found in the book by Lynch and 
Livingston (see Further Reading). 


Reflection by metals: So far I have limited 
the discussion to reflection from noncon- 
ductors. The problem with conductors of 
electricity is that they absorb light and, in 
some cases, absorb very strongly. Actually, 
the stronger the absorption, the higher the 
reflectivity, and the reflectance can reach 
nearly 100% without any polarization for 
polished metals. More poorly reflecting met- 
als such as steel partially polarize reflected 


light. For a typical metal, reflectance of both 
TM and TE beams depends on the angle of 
incidence, and the TM beam shows mini- 
mum, though nonzero, reflectivity at the 
“principal angle of incidence” There are no 
universal polarization curves for metals since 
the reflection of polarized light depends on 
the optical absorption coefficient, which in 
turn depends on the wavelength of the light. 
Photographers must depend on trial-and- 
error methods with metals. 


13.5 UV and IR Photography 


he human eye can only see a very 
ninos band of radiation between 

about 400 nm and 700 nm that is 
defined as the visible region. Thus, we are 
restricted to a region of the electromagnetic 
radiation spectrum where the frequency 
varies by only a factor of two. At first glance 
that seems remarkable, since the complete 
spectrum from gamma rays to radio waves 
encompasses almost 15 orders of magni- 


tude change in frequency. However, humans 
evolved the ability to see sunlight; and most 
of the energy from the sun lies in the visible 
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region. The solar flux is illustrated in Figure 
13.17. At wavelengths less than 400 nm, 
where the UV-A region begins, the inten- 
sity of the solar spectrum decreases, and 
it is severely attenuated below about 320 
nm because of absorption by ozone in the 
atmosphere. This loss of intensity is a very 
good thing since photons in the UV region 
have high energy and can damage skin cells 
as well as the photoreceptors in the eye. In 
the infrared region beyond 700 nm there is 
still considerable intensity, but the photons 
have low energy and the photoreceptors 
in the eye are not very efficient at their 
detection. 


Obtaining UV images: So we accept the fact 
that human vision is not good in the UV and 
IR regions, but what about photography. 
First consider the near UV region (320-400 
nm, UV-A). The major problem we encoun- 
ter is that glass in standard lenses begins 
to absorb light at about 420 nm and by 370 
nm transmits very little. Practitioners in 
UV photography use special quartz/fluorite 
lenses that transmit UV at least to 300 nm. 
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The UV-Nikkor 105 mm lens is often men- 
tioned, and the Jenoptik CoastalOpt 60 mm 
J/4 Apo Macro is a special purpose lens that 
covers the region from 310 nm to 1,100 nm. 
Both of these lenses are extremely expensive. 
Some commonly available lenses transmit 
light well beyond 400 nm and permit ama- 
teur photographers to experiment with near 
UV photography. The best candidates are 
simple lenses without cemented elements 
and without optical coatings, and adapted 
enlarger lenses also have been suggested. 

The next thing that is necessary for UV 
photography is a filter that can completely 
block visible light. The Kodak Wratten 18A 
transmits from 300 nm to 400 nm and blocks 
visible light, but it also transmits IR from 
700 nm to 900 nm. Other UV transmis- 
sion filters such as the B+W 403, which are 
less expensive, have the same IR problem. 
There are better but much more expensive 
UV pass filters. The Baader U filter trans- 
mits UV light in the region 320-390 nm and 
otherwise blocks the region 220-900 nm. 
Also, LifePixel and other companies offer 
UV- specific camera conversions. 


700 900 1000 


FIGURE 13.17. Sketch of the solar intensity versus wavelength above the atmosphere and at 


sea level. 
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(b) 


FIGURE 13.18. Paul Green Cabin: Canon G-1 camera (a) without filter and (b) with B+W 403 UV 


pass filter. 


Finally, one needs a sensor that can detect 
UV. CMOS sensors in most digital cameras 
turn out to be satisfactory. It is even possi- 
ble to obtain interesting results with some 
standard lenses and some experimentation. 
As an example, I show images obtained 
with a Canon G-1 camera with and without 
a B+W 403 UV transmitting filter (Figure 
13.18). As noted, the filtered image is con- 
taminated with IR radiation. The UV image 
was brightened with Photoshop, but the 
false colors were a surprise. It is fair to say 
that UV spectroscopy is not of much inter- 
est to amateur photographers except maybe 
in astrophotography. The important uses of 
UV photography include dental imaging, 
skin damage analysis, and the recovery of 
evidence in forensic science. 


Obtaining IR images: Fortunately, IR pho- 
tography is fairly easy and often rewarding 
for amateurs. Although film with IR sensi- 
tivity is increasingly difficult to find since 
Kodak discontinued production, Rollei and 
Efke IR films may still be available. Lenses 
are not a big problem in the IR. Many 
standard lenses are satisfactory for IR pho- 
tography. The problem, when it arises, is 


that some lenses give “hot spots” in IR pho- 
tographs. Also, the focus scale shifts and 
some lenses lose resolving power especially 
at the edges. One must select lenses by trial 
and error. There are also lists of lenses that 
have been found to be satisfactory, but I find 
some of the recommendations not com- 
pletely accurate. Some high quality lenses 
fail to give corner-to-corner sharpness with 
IR light. 

Of course, filters have to be used to 
block visible light, and here there are a lot 
of choices. One has to decide on the desired 
cut-off point. Some popular filters are 
listed in Table 13.2. Many other filters are 
available, and equivalent filters can often 
be obtained from B+W, CoCam, Cokin, 
Heliopan, Hoya, Schott Glass, and Tiffen. 

In digital photography we must deal 
with CCD or CMOS sensors in place of 


TABLE 13.2. 
Red and IR filters 
Kodak Wratten filter 25 89B 87 87c 
0% Transmission (nm) 580 680 740 790 
50% transmission (nm) 600 720 795 850 
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film. These sensors are inherently sensi- 
tive in the red/IR region, so much so that 
without blocking ambient IR the recorded 
colors may be distorted, i.e. IR will be will 
falsely recorded as a visible color. Therefore, 
manufacturers have installed hot-mirror 
IR reflectors to prevent IR radiation from 
reaching the sensors. Of course, the hot 
mirrors are not perfect and some IR gets 
through. So we add a filter to block visible 
light and hope that enough IR gets through 
the hot mirror to permit an image to be 
recorded with a reasonable exposure time. 
As you might guess, some cameras are acci- 
dentally better than others at producing IR 
images this way. As an example, I used a 
Canon G-1 (3 megapixel) with a Hoya R72 
(Wratten 89B equivalent) on a tripod to 
photograph Mabry Mill on the Blue Ridge 
Parkway. Figure 13.19 shows this photo- 
graph compared to one taken (hand-held) 
without the IR filter. Both were converted 
to monochrome in Photoshop. The obvious 
differences are in the foliage, which is white 
in the IR photo, and the water, which is black 
in the IR photo. Blue sky would also be very 
dark in an IR image, but in this case there 


were thin clouds. In general, IR images are 
more striking than ordinary black and white, 
and it is possible to obtain very interesting 
false color images. Even though the expo- 
sure time was very long for the IR shot, the 
experience was not bad because the image 
was Clearly visible on the LCD screen even 
though the filter looked quite opaque. 

Of course, action photos are out of the 
question with long exposures, and even a 
breeze can upset things by moving leaves 
and branches. The way around this prob- 
lem is to have the IR filter (reflecting hot 
mirror) removed from the digital camera. 
Several companies offer this service now 
for both point-and-shoot and interchange- 
able lens cameras. There are two options. 
The IR filter can simply be removed, and 
the user then must use an external filter for 
normal photography and a visible blocking 
filter for IR photography. The downside of 
this arrangement for DSLRs is that one can- 
not see through the dark IR filter and must 
remove the filter to compose the scene. The 
other option is to have the IR blocking filter 
(hot mirror) in the camera replaced with 
a visible blocking filter mounted directly 


FIGURE 13.19. Mabry Mill: Canon G-1 camera (Upper) without a filter and (Lower) with a Hoya 


R72 IR filter. 
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on the sensor. There are more possibilities 
here because the available blocking filters 
range from 590 nm (super color) to 720 nm 
(standard IR) and even to 830 nm (deep IR). 
The LifePixel website offers useful tutorials 
about filter choice and image processing.” 

With filter replacement, the camera 
works normally and is even convenient for 
action photography. The IR conversion is 
an attractive option for a camera that has 
been replaced with a newer model. There 
is, however, a change in the focusing scale 
that requires compensation unless focusing 
is based on the image at the sensor. This is 
not a problem for DSLRs in Live View mode 
or for mirrorless cameras. Companies that 
advertise IR conversions include LifePixel 
and MaxMax. 

The images shown in Figure 13.20 were 
obtained with an 18 megapixel Canon SL1 
(100D) camera that had been converted to 
IR by LifePixel. A 590 nm filter was chosen 


so that some color information could be 
retained. The images were taken at mid- 
day with a wide-angle lens (1/320 s, //10, 
ISO 200). Processing of the black and white 
image consisted of minor adjustments to 
exposure, contrast, clarity, and sharpness 
followed by conversion to monochrome 
in Lightroom. The work flow for process- 
ing of the color image was more involved. 
First, the white balance was adjusted either 
with Canon Digital Photo Professional soft- 
ware or with a custom camera profile in 
Lightroom. The image was then exported to 
Photoshop so that the Channel Mixer could 
be used to switch the blue and red channels 
to get blue color into the sky. The positions 
of the red, blue, and green color sliders are 
arbitrary and are set according to the pho- 
tographer’s preference. Recommendations 
about IR image processing can be found 
in LifePixel tutorials and Bob Vishneski’s 
website.€ 


FIGURE 13.20. Point Reyes shipwreck IR image rendered in black and white and in color. 
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Fowles, G. R. (1989). Introduction to Modern 
Optics, 2nd Ed. New York: Dover Publ. A 
good modern optics text. 

Land, E. H. (1951). Some Aspects of the 
Development of Sheet Polarizers. 
Journal of the Optical Society of America, 
41(12), p. 55. Dr. Land discusses the his- 
tory of polarizers. 

Lynch, D. K. and Livingston, W. (2001). 
Color and Light in Nature, 2nd Ed. 
Cambridge: Cambridge University Press. 
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Notes 


1. Filters have a long history in photogra- 
phy. The earliest photographic emulsions 
were primarily sensitive to blue and UV 
light, but in the 1870s the German pho- 
tochemist and photographer Hermann 
Vogel learned how to extend the sen- 
sitivity into the green region by adding 
dyes to silver bromide. The result was 
the orthochromatic plate, still lacking 
sensitivity to red light. Adolph Miethe, 
an astrophotographer who succeeded 
Vogel as Professor of Photography at the 
Royal Technical University in Berlin, was 
able to extend sensitivity to the complete 
visible spectrum in 1903 to produce 
panchromatic emulsions through the 
addition of the sensitizing dye, ethyl red. 
However, there was still excessive sen- 
sitivity in the blue region, and realistic 
tones could only be achieved by photo- 
graphing through a yellow filter. 


Photographic filters that selectively 
attenuate colors were invented by 
Frederick Wratten for dealing with the 
limited tonal ranges of available emul- 
sions. The ability to modify tones in 
black and white images became very 
important to photographers, and in 1912 
Eastman Kodak purchased Wratten’s 
company and continued the produc- 
tion of “Wratten” filters. Later, after 
color emulsions became available, filters 
were even more important. Each color 
emulsion was designed with a particular 
white balance. For example, film for day- 
light use might be balanced for 5500K 
light. Such an emulsion would produce a 
decidedly yellow-orange cast if used with 
tungsten incandescent light (3400K). 
The way around the problem was either 
to purchase a film balanced for 3400K 
light or to use a blue filter (Wratten 80B). 
Conversely, “tungsten” film balanced for 
3400K could be used in daylight by add- 
ing an amber filter (Wratten 85). 

The introduction of digital cameras 
along with computer post-processing 
of images has drastically reduced the 
need for colored filters. These cameras 
have built-in UV and IR filters to avoid 
false colors, and the white balance can 
be set in the camera or (in some cases) 
postponed for later processing. Overall 
attenuation to permit long exposures 
is still desirable, and selection of the 
direction of polarization remains an 
important option. In addition, ultravio- 
let (UV) and infrared (IR) photography 
require that visible light be blocked with 
a filter. 


. Unfortunately, Bouguer did not get full 


credit for the absorption law because he 
did not provide a compact mathematical 
expression to describe the effect. Johann 
Lambert’s contribution was to apply the 
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calculus to sum up the contributions of 
infinitesimally thin layers to show that 
the attenuation decreases exponentially 
as described by the equation J/ J, = 107 
02 where OD is defined as the optical 
density. [Actually, Lambert’s expression 
was closer to J = [,exp(—a - I) where a 
is a constant that denotes the absorp- 
tion coefficient of the material and / is 
the thickness of the sample. There is a 
simple conversion between the forms of 
the equation, given by OD = a - !/In(10).] 
So how did August Beer get into the act? 
His contribution was to relate the OD, or 
absorbance as chemists call it, to the con- 
centration of particles or molecules in a 
transparent liquid solvent or perhaps the 
atmosphere. Thus, in place of Lambert’s 
constant a, Beer introduced the prod- 
uct of a molar absorption coefficient 
and the concentration of the particles. 
This turned out to be very important 
for chemists because, given the molar 
absorption coefficient, the concentration 
of a solution could be determined from 
a measurement of the optical density (or 
absorbance). 

. Some forms of crystalline tourmaline 
are naturally dichroic so that one crystal 
will polarize light and a second crystal 
can be rotated to totally block the polar- 
ized light. However, tourmaline absorbs 
light too strongly to be used as a prac- 
tical polarizer. Crystals of herapathite 
(quinine sulphate periodide) were also 
found to be dichroic in the nineteenth 
century, but it turned out to be very diffi- 
cult to grow the crystals large enough to 
make useful polarizers. In spite of that, 
Carl Zeiss was able to market “Bernotar” 
single crystal herapathite polarizing fil- 
ters for photography. Information about 
the crystal growing techniques they used 
was apparently lost in World War II. 


Commodity polarizers date from 
the 1920s, when the scientist/inven- 
tor Edwin Land demonstrated that a 
suspension of tiny crystals (ground in 
a mill) aligned by electric or magnetic 
fields can also polarize light. With the 
insight that aligned crystallites or even 
aligned molecules can substitute for 
a single crystal, he went on to develop 
polarizing sheets of PVA (polyvinyl 
alcohol) doped with iodine atoms. The 
trick was that the sheets were stretched 
in the manufacturing process in such a 
way that the polymer molecules were 
aligned and the attached iodine atoms 
formed chains. The chains of atoms act 
like short segments of metallic wire that 
strongly absorb light polarized paral- 
lel to their length. It is the electrons, 
that are free to move in the direction of 
the atomic chain, that interact with the 
electric field With this technique and 
various improvements, Land’s company, 
Polaroid Corporation, was able to pro- 
duce efficient and inexpensive polarizers 
for sunglasses, photography, and a variety 
of other uses. 


. Suppose that polarized light with the 


maximum amplitude E and the intensity 
I (Figure 13.11) is incident on a linear 
polarizer so that the component E, is 
transmitted and E, is blocked. The trans- 
mitted light has the amplitude E, = E: cos 
09 and the intensity is J, = J - cos’@. If the 
incident light is unpolarized, the angle 0 
is distributed over 360°, or 2m radians, 
and one must obtain an average of cos?0 
in order to compute the intensity of 
the transmitted light. The result for the 
transmitted intensity is Z zea = 7/2. This 
explains why 50% of unpolarized light is 
transmitted by a linear polarizer. 

The expression for the circularly 
polarized light is more interesting. At an 
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instant in time, say time 0, the amplitude 
of plane polarized light varies according 
to position on the z-axis (see Figures 
13.8-13.10). For the x-component the 
amplitude is described by E, = cos(k - z) 
where k = 211/1. If the y-component is 
represented by E, = cos(k- z) with E, = E, 
then the result is just a linearly polarized 
wave at 45° as shown in Figure 13.11. 
However, the introduction of a quarter- 
wave plate changes things so that the 
y-component becomes E, sin(k - z) and 
the resultant rotates around the z-axis. 


Note that a quarter-wave plate requires 
no filter factor because the electric field 
of the light beam is not attenuated. In 
fact, a second quarter-wave plate (rotated 
by 90°) can be used to restore the origi- 
nal linear polarization without loss. The 
x- and y-components shown here were 
used in the program Mathcad to gener- 
ate the sine curves in Figures 13.9 and 
13.10 and the helix in Figure 13.10. 


. https://www.lifepixel.com/tutorials 
. https://photographylife.com/how-to- 


process-infrared-photographs#comments 


Taylor & Francis 
Taylor & Francis Group 


http://taylorandfrancis.com 


CHAPTER 14 


The Limits of Human Vision 
(How Good Does a Photographic Image Need to Be?) 


The eye is the jewel of the body. 
—HENRY DAVID THOREAU 


You can observe a lot just by watching. 


—YOGI BERRA 


14.1 Introduction 


hotography is a human activity that 
Peere entirely on the capabilities 

and limitations of the human visual 
system. We judge the apparent resolution, 
brightness, contrast, and the quality of 
color in photographs; but these attributes 
are subjective since everything depends 
on perception. Our vision results from a 
combination of physiology and psychology 
which correspond roughly to the eye and 
the brain. Of course, all eyes are not the 
same, but a typical or standard observer 
must be assumed to exist by those design- 
ing photographic equipment. Therefore, it 
is necessary to consider what the typical 
healthy eye can see. A study of the eye can 
provide us with criteria for image resolu- 
tion and color stimuli that humans can 
distinguish and appreciate in photographic 
images. The perception of color, resolu- 
tion, and overall image quality, of course, 


depends on the wiring of nerve cells in the 
retina and the visual cortex. Consideration 
of this part of the visual system can be 
found in Chapters 15 (color), 17 (resolu- 
tion), and 18 (art appreciation). Of course, 
many individuals deviate strongly from the 
standard observer. Color vision anomalies 
are discussed in Appendix F (Observing and 
Managing Color: Dealing with Color Vision 
Anomalies). 


14.2 Structure of the Eye 


irst, we look at the structure of the 
Fe: The eye has some of the features 

of a camera, or better yet a TV cam- 
era. The relevant components are shown in 
Figure 14.1. On the left, we find the cornea 
and the lens which work together as a com- 
pound lens. Most of the focusing power 
(about 40 diopters) results from refraction 
at the air/cornea interface, while the lens 
provides adaptive focus or accommodation 
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FIGURE 14.1. Simplified schematic of the eye. 


over a range of about 20-30 diopters. With 
a relaxed lens, the focal length of the com- 
bination is about 22 mm taking into account 
the refractive index of the vitreous humor 
(n = 1.33). Immediately in front of the lens, 
we find the iris or diaphragm that controls 
the size of the pupil. The iris is the colored 
part of the eye, and the pupil appears to be 
black. The pupil ranges in diameter from 
less than 2 mm to more than 8 mm, so that 
the optical system can be described as about 
J/10 to f/2. 

The projected image falls on the retina 
shown at the right in Figure 14.1. The ret- 
ina is a structure about 0.5 mm thick that 
covers the back of the eye and contains the 
photoreceptors. The top layer (facing the 
lens) is made up of ganglion or output neu- 
trons, the middle region consists of nerve 
tissue, and the bottom layer contains rods 
and cones (the visual photoreceptors). The 
central layers of nerve tissue are quite com- 
plex and contain networks of horizontal 
cells, bipolar cells, and amacrine cells con- 
necting the photoreceptors to the ganglion 
cells. Behind the photoreceptors is a black 


layer, the pigment epithelium, that absorbs 
light and limits back reflection. A sche- 
matic of structure of the retina is shown in 
Figure 14.2. Overall the structure appears to 
be upside down. Light has to pass through 
multiple layers, where it is attenuated and 
dispersed before it reaches the receptors. 
This arrangement is certainly expected to 
limit visual resolution. However, the struc- 
ture of the retina is not uniform, and there is 
a special region (described in Section 14.3) 
that permits high acuity (visual sharpness). 

In addition to the well-known rods and 
cones, it has been discovered that that there 
is a fifth type of photoreceptor in the eye. It 
turns out that some of the ganglion cells are 
photoreceptors as well as neurons and are 
part of an independent system that moni- 
tors the level of illumination for control of 
physiological functions including phasing 
of the circadian clock and control of pupil 
size. These ganglion cells (ipRGCs) respond 
to blue light (460-500 nm) and may make 
use of melanopsin as the photopigment. 
However, this system is apparently not 
directly related to vision and is not relevant 
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FIGURE 14.2. Schematic diagram of the retina. 


to our consideration of the eye as an optical 
instrument.’ 


14.3 Operation of the 
Visual Receptors 


o understand how the eye accom- 

plishes high resolution and color 

vision as well, one needs to consider 
the visual receptors. There are about 130 
million receptors in the retina. Of these, 
about 120 million are rods that are highly 
sensitive receptors containing the visual 
pigment rhodopsin. A rod can detect a sin- 
gle photon. The rods respond to blue-green 
light and have maximum sensitivity at about 
500 nm. They are responsible for night 
vision but permit no resolution of color. 
In addition, there are 6 or 7 million cones 
which contain visual pigments called opsins. 
Experiments have demonstrated that cones 
come in three varieties commonly known 
as red, green, and blue cones. Actually, each 
type of cone responds to a wide range of fre- 
quencies, but their frequencies of maximum 
absorption are shifted as follows: 564 nm 
(red or long wavelength L-cones), 534 nm 


(green or medium wavelength M-cones) 
and 420 nm (blue or short wavelength 
S-cones). The normalized absorption spec- 
tra for human photoreceptors are shown in 
Figure 14.3. 

It turns out that our visual acuity and 
color vision depend on a special region of 
the retina, known as the fovea centralis or 
fovea for short, that is about 0.4 mm (400 
um) in diameter. The entire region respon- 
sible for high acuity vision including the 
foveal pit and the foveal slope (perifoveal 
and parafoveal regions) is known as the 
macula region. In illustrations of the ret- 
ina, the fovea is usually shown as a notch 
or pit in the surface because the neuron 
layers are pushed out of the way in this 
small area to expose the photoreceptors to 
unobstructed photons from the lens. This 
structure is illustrated in Figure 14.4. The 
fovea contains approximately 30,000 cones 
in a tight mosaic pattern and no rods. In the 
fovea, the cones have diameters of 1-4 um 
while elsewhere in the retina, where their 
density is much lower, their diameters are 
4—10 um. The rods are distributed through- 
out the retina but their maximum density 
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FIGURE 14.3. Absorption spectra of human cones (adapted from Bowmaker and Dartnall, 1980).? 
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FIGURE 14.4. Schematic of the macula region; the rods are shown in purple and the RGB seg- 


ments are cones. 


is found at about 4-5 mm from the fovea. 
Both rods and cones are more or less rod- 
shaped with lengths of approximately 50 um 
perpendicular to the surface of the retina. 
Another way the fovea gets special treat- 
ment is that the ratio of cones to ganglion 
cells is only 3:1, while elsewhere in the retina 
the ratio of receptors to output neurons is 
125:1. 


The rods are much more sensitive to 
light than the cones, but in bright light they 
are saturated and do not function. In dim 
light the cones are not effective, but as light 
intensity increases the cones take over and, 
furthermore, provide information about the 
distribution of wavelengths in the stimulus 
that permits colors to be distinguished. 
Surprisingly, the three types of cones are 
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not present in anything close to equal 
amounts in the fovea. The blue (S) cones, 
which are somewhat larger in diameter and 
more sensitive than the red (L) or green (M) 
cones, only comprise about 3% of the cones 
in the fovea. How the excitation of these 
cones leads to color vision is an interesting 
but incompletely understood subject, that I 
will return to in Chapter 15. 


14.4 Visual Acuity 


JE acuity of the eye as an optical 
instrument is the easiest part to 
understand. As shown in Figure 
14.4, the fovea with a diameter of about 0.4 
mm subtends an arc of about 1* as required 
by the equation, arc length = (focal length) 
x angle, where the focal length is 22.3 mm. 
This represents the angular range for high 
acuity vision and is about twice the angle 
subtended by the sun or moon. This fovea 
also corresponds to the area of 30,000 cones 
each having a radius of about 2 um. Since 
the resolution of points or closely spaced 
lines depends on cones whose centers are 
separated by about 4 um in the fovea, the 
next question is what angle of view corre- 
sponds to this spacing. By using 4 um as 
the arc length of a cone we find an angle 
of about 0.6 minutes of arc. This is close 
to the value of (1/60) deg = 1 min that is 
usually assumed for visual acuity in photog- 
raphy and for 20/20 vision.’ To put this in 
perspective, I note that a dime (18 mm) at 
200 ft. subtends an angle of approximately 
1 arcmin. 

Of course, the question of resolving 
power for the eye or any other lens system 
is more complex than I have described. 
Suppose that light comes from a point 
source, say a Star, and the angle subtended 
is close to zero. The image projected on the 
retina has a width that depends on the size 


of the pupil and aberrations in the cornea 
and lens. When the aperture (pupil) is 1 
mm, the spot size and intensity pattern are 
determined primarily by diffraction as in a 
pinhole camera. For large pupil sizes aber- 
rations dominate, and the optimum pupil 
size for resolution is between 3 mm and 
5 mm. In general, the width of the point 
spread function (PSF) indicates the quality 
of the optical system. For straight line light 
sources the line spread function (LSF) gives 
similar information. 

Another way to explore resolving power 
is to measure the eye’s response to a pat- 
tern of equally spaced white and black lines 
as a function of their spacing, i.e. line pairs 
per degree (lpd). This kind of test can also 
be performed with a sinusoidal pattern 
of white peaks (maxima) and black val- 
leys (minima), in which case the spatial 
frequency is denoted by cycles per degree 
(cpd). The idea is to measure the contrast 
of the image on the retina and to compare 
that with the contrast of the original object 
pattern. If the peaks and valleys in the image 
have the intensities Z and» respectively, 
the contrast of the image is defined as: 


C Lss ii Tain 
a I be + Lain 


ma: 


The ratio of the image contrast C, to that 
of the original stimulus C, then defines the 
modulation transfer function for the par- 
ticular spatial frequency: 


C 
MTF(v) = E 


0 


where v denotes the spatial frequency in 
line pairs or cycles per length or degree. The 
function MTF(v) is mathematically related 
to the PSF. What we find is that the apparent 
contrast decreases as the spatial frequency 
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increases until only a uniform gray with 
zero contrast can be seen. This procedure 
is illustrated by the stimuli and responses 
in Figure 14.5. 

The MTF is often used as a measure of 
optical quality, and the frequency at which 
the MTF drops to 0.5 is often taken as an 
indicator of lens resolution. It definitely 
provides information about acuity, but it 
does not give complete information about 
the phase transfer function. In general, the 
MTF should be measured in both the tan- 
gential and the sagittal directions. The MTF 
for the human eye at various pupil sizes is 
shown in Figure 14.6. These curves were 


5 cpd 


20 cpd 


computed with the equations reported by 
Rovamo, et al.* It was concluded that the 
eye is close to a diffraction-limited optical 
system at a pupil size of 1 mm but is con- 
siderably worse than diffraction-limited at 
larger pupil sizes. 

It is important to point out that Figure 
14.6 describes the optical system of the eye 
and not the MTF for the complete visual 
system. Psychophysical experiments reveal 
that humans have most sensitive response 
to frequencies close to 6 cpd and that 
response depends strongly on contrast. 
When the contrast sensitivity is taken into 
account the complete MTF has a maximum 


40 cpd 


max 


FIGURE 14.5. Stimuli (black) and responses (red). 
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FIGURE 14.6. Foveal optical MTF for the human eye for various pupil sizes. 


close to 6 cpd and falls off at both high and 
low frequencies. The high-frequency region 
is dominated by the optical properties that 
are responsible for the curves in Figure 14.6 
while the low-frequency part depends on 
the properties of frequency sensitive cells 
in the visual cortex. For a more complete 
discussion of this point, see Chapter 17. 

A comparison of the resolution of the 
eye and the digital camera also involves the 
estimation of the effective number of photo- 
receptors (pixels) in the eye.? We have seen 
that the central fovea (fovea) subtends only 
about 1° of arc in the field of view (FOV), 
but that it seems to provide visual acuity 
everywhere by scanning as needed. This is 
somewhat analogous to a high-resolution 
film scanner which only has a single row 
of detectors, but which captures details 
from an entire negative by scanning. We 
ask now how many cones (sensors) would 
be required in the retina for the eye to be 
able to capture the complete FOV simul- 
taneously as electronic sensors do. This 
calculation is easy after we decide what 
the FOV is. The eye has a total horizontal 
FOV of at least 180°, but our impression of 


the high-resolution field is much narrower. 
I am going to assume arbitrarily 120° and 
80° for the horizontal and vertical FOV’s, 
respectively. This angular rectangular cor- 
responds to a solid angle of 2.7 steradians 
(sr), while the fovea subtends only 2.4 x 
10% sr. (As a reference, 4 is the solid angle 
for all space. Please visit “Solid Angles” in 
the Wikipedia entry for an explanation.°) 
Therefore, the rectangular FOV has an area 
of about 10,000 foveas; and, since each fovea 
contains 30,000, cones we conclude that the 
number of effective pixels is 300 million. So 
digital cameras have a way to go; but they do 
provide detection of color at low light levels, 
no delay for dark adaptation; and, of course, 
the ability to transfer and store images. 


14.5 Sensitivity 


T sensitivity of the eye is another 
property that can be compared with 
a digital camera. In bright light the 
cones are effective (photopic vision), and at 
very low light levels the rods take over (sco- 
topic vision), giving much higher sensitivity. 
The effect is something like changing the 
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ISO sensitivity of a camera sensor except 
that dark adaptation of the eye can require 
up to 45 minutes. Psychophysical evidence 
shows that the cones and rods adapt inde- 
pendently as light conditioned eyes adapt to 
total darkness.”* In these experiments the 
subject reports the lowest level of light that 
can be detected on an illuminated patch 
during the adaptation period. The curves 
in Figure 14.7 illustrate how the sensitiv- 
ity of the cones decreases for eight to ten 
minutes before reaching a plateau. During 
this period of cone mediated adaptation 
the sensitivity threshold decreases by a 
factor of up to 100. The sensitivity thresh- 
old for the rods also decreases and, after 
the junction of the cone and rod curves, 
rod adaptation continues for another 30 
minutes. 

The sensitivity thresholds in Figure 14.7 
are typical of those for a middle-aged sub- 
ject undergoing dark adaptation after a few 
minutes of pre-adapting light. For older 
subjects the thresholds are higher and the 
lowest dark adaptation value for an 80-year- 
old is a factor of two higher than that for 
a 20-year-old. Dark adaptation is faster for 
lower levels of pre-adapting light; and for 


short pre-adapting durations (less than a 
several minutes) the bi-phasic cone and rod 
branches of the curve do not develop. 

Mesopic vision occurs in situations when 
light stimulates both cone and rods. Any 
light cast on the fovea will be detected if it 
exceeds the adaptation threshold for cones, 
while light cast on the peripheral areas will 
detected by rods. There is only one type of 
rod, so there will be no resolution of color 
in the peripheral vision. Furthermore, the 
luminosity functions of the human eye 
show that the peak sensitivity for rods 
(scotopic vision) is shifted to shorter wave- 
lengths relative to that for cones (photopic 
vision) and, in fact, rods detect almost no 
red light. Dark adaptation also depends on 
the wavelength of the pre-adapting light. 
Therefore, a red light may have less effect 
on dark adapted eyes than a blue light 
and in any event is known to be less dis- 
turbing to nocturnal animals than white 
light. Also, the fovea contains red cones 
and no rods, and red light provides high 
acutance. Rod vision provides only low- 
resolution vision and can only be accessed 
by looking slightly away from the intended 
subject. 
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FIGURE 14.7. The sensitivity threshold is plotted versus time in the dark. 
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Mesopic vision is experienced at sunset, 
in twilight, under a full moon, and in many 
urban environments with artificial lighting. 
The cones are always active and even with 
dark adapted eyes in total darkness a small 
light source in the distance will be seen with 
color if there is high enough luminance. This 
idea is confirmed by the apparent color tints 
of planets. The Milky Way is another ques- 
tion. Under dark skies in remote places the 
Milky Way presents a beautiful spectacle of 
bright stars and unresolved clouds of stars; 
however, we see no color. Unfortunately, 
there is only enough illuminance for rod 
vision. Figure 14.8 shows the Milky Way that 
was photographed at Acadia National Park. 
The camera sensor responds to red, blue, 
and green light and reveals what we think 
humans would see if their vision threshold 


was thousands of times lower. Here again 
photography lets us see the world in a dif- 
ferent way as we do with IR, UV, and black 
and white photographs. 

Clark estimated the ISO equivalent of the 
dark adapted eye by comparing the ability of 
the eye to see magnitude 14 stars through a 
5 in. aperture lens with a Canon 10D digital 
camera exposing the image for 12 seconds in 
the same situation, and he concluded that the 
eye offers about ISO 800 sensitivity." In bright 
light where the rods saturate and the cones 
take over, the sensitivity has been estimated 
to be about ISO 1. An important difference 
between the retina of the eye and the sensor 
in the camera is that the eye always has low 
sensitivity cones and high sensitivity rods 
in place, while the camera sensor resets all 
detectors to the same ISO sensitivity. 


FIGURE 14.8. Photograph of the Milky Way showing colors that are not visible to the unaided eye. 
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There is a tendency to compare rods in 
the human eye with detectors in a sensor 
and to ask if a rod can detect a single pho- 
ton. The answer appears to be yes, but the 
observer would not be aware of the event. 
A conscious response requires the rods to 
detect several photons in a short period of 
time. The threshold for detecting light fall- 
ing on 10 minutes of arc in the region of 
highest concentration of rods on the retina 
is about 90 photons. Considering losses 
and the number of rods involved, the rods 
must be responding to single photons. This 
experiment emphasizes the fact that in the 
visual system everything depends on the 
wiring of the photoreceptors and the firing 
of nerve cells. 

The really impressive thing about the 
visual system is how the signals from 130 
million receptors in two eyes can be pro- 
cessed to give useful images in real time. 
This brings us to questions about how sig- 
nal processing is distributed between the 
retina and the visual cortex in the brain and 
basically how the brain works. While that is 
beyond the scope of this book, I will treat 
the major theories of color vision in the next 
chapter and ideas about visual perception in 
Chapters 18 and 19. 


14.6 Conclusions 


has similar optical properties to a small 

point-and-shoot digital camera, except 
that it only has to provide a high-resolution 
image with a field of view of about 1°. It can 
do this with an equivalent of only about 
30,000 pixels (cones) in the fovea because 
the point of interest is always directed at that 
region. The effective pixel diameter of about 
4 um limits the acuity of the eye to about 1 
arcmin and provides some justification for 
our choice of the CoC in Chapter 11. 


[= shown in this chapter that the eye 


Furthermore, the cones provide three 
color receptors and justify to some extent 
our use of three colors to generate all the 
colors in color displays. However, as we 
shall see, it is not possible to synthesize all 
the colors we can perceive by mixing light 
from three narrow band (monochromatic) 
sources. Similarly, our electronic sensors do 
not have the same frequency distribution 
of sensitivities as human cones. Detection 
and display systems based on three pri- 
mary colors can provide a good, but not 
perfect, representation of the colors our 
eyes respond to. Obtaining an image that 
matches the appearance of the original 
scene is a more challenging problem. The 
trichromacy color theory is discussed in 
Chapter 15 along with the alternative oppo- 
nent theory. Other aspects of the visual 
system are presented in Chapters 17, 18, 
and 19. 
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CHAPTER 1 5 


How Can Color Be Managed? 


For the newly sighted, vision is pure sensation 
unencumbered by meaning. 


—ANN DILLARD 


But, to determine more absolutely, what Light is, after what 
manner refracted, and by what modes or actions it produceth 
in our minds the Phantasms of Colours, is not so easie. 
And I shall not mingle conjectures with certainties. 
—ISAAC NEWTON 


15.1 Introduction 


uman color vision is possible 
because three types of cones (S, 
M, and L) in the retina respond 


to slightly different ranges of wavelengths 
of light. This part is well understood in 
terms of chemistry and physics, but it only 
represents the first steps in a truly amaz- 
ing process. The perception of color and, in 
fact, the complete illusion of a multicolored 
world is generated in the visual cortex of the 
brain. Information about the distribution 
of intensities at various wavelengths avail- 
able from the three types of photoreceptors 
is first processed by bipolar and ganglion 
cells in the retina and is then transmitted 
via axons in the optic nerve to the laternal 


geniculate nucleus (LGN). In the LGN sig- 
nals from the two eyes are integrated and 
projected directly to the primary visual 
cortex (V1), which occupies the surface of 
the occipital lobe of the cerebral cortex (see 
Figure 15.1). 

In the last few decades of the 20th 
century, noninvasive imaging methods 
became available for the study of normal, 
healthy human brains. In particular, func- 
tional magnetic resonance imaging (fMRI) 
permits brain response to visual stimuli 
to be correlated with specific locations 
in the visual cortex. These developments 
have prompted a large number of studies 
of localized brain function. The consen- 
sus view is that the visual cortex contains 
a set of distinct visual field maps, and that 
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FIGURE 15.1. A sketch of the human brain showing the lobes (blue type), the primary visual 
cortex (V1) and other specialized visual areas as described by S. Zeki in /nner Vision (1999). 


nearby neurons analyze nearby points in the 
visual field. While the cells in V1 appear to 
respond to wavelength, the perception of 
color requires additional processing. Visual 
maps have been established in the visual 
cortex areas V1, V2, and V3; 20-30 visual 
areas have been distinguished, and the fol- 
lowing picture has emerged. Information is 
transmitted via two pathways. In the dorsal 
stream information passes from V1 through 
V2 and on to the middle temporal area (MT 
or V5). This is the “where pathway” that 
is concerned with location and motion of 
objects and control of the eyes. The ventral 
stream or “what pathway” also begins with 
V1 and V2, but directs information to the 
visual area known as the V4 complex and 
the adjacent V8 area. This path is involved 
with representation, long-term memory, and 
the association of color with visual stimuli. 
The exact location of visual processing 
is not of primary concern here, but rather 
the fact that neural processes deep in the 
visual cortex of the brain are the origin of 
color! Objects don’t have color, and nei- 
ther do light beams. Objects simply absorb, 
transmit, and reflect various amounts of 


incident electromagnetic radiation and the 
radiation is characterized by wavelength 
and intensity. If radiation falls in the range 
of about 400 nm to 700 nm, the photorecep- 
tors in our eyes can respond to it (see Figure 
14.3). The response is processed and used in 
various ways in analyzing the visual field for 
patterns and motion, but the association of 
color with the excitation of cones in the ret- 
ina is another thing altogether. Part of this 
picture has been revealed through the study 
of defects in color vision. (See Appendix F). 
Color vision deficiency (dyschromatopsia) 
is often genetic in nature. One or more of 
the three types of cones (S, M, or L) may 
be missing or defective. Thus, about one in 
12 males and a smaller fraction of females 
suffer from some color vision deficiency, the 
most common of which is red-green color 
blindness. Each deviation in function of a 
cone generates a different perception of the 
world around us, and it is clear that a shift in 
the region of maximum sensitivity of all the 
cones and rods by a few hundred nm would 
present to us a strange, alien world with 
little similarity to the one we now accept as 
reality. 
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Now suppose that color perception is 
destroyed by an injury or a disease that 
affects visual areas in the cortex, perhaps 
the V4 complex. Such events are, in fact, 
well documented and the consequences 
are more dire than might be imagined from 
the simple loss of color for the visual field. 
Cerebral achromatopsia patients experi- 
ence a drab and disturbing world with 
little similarity to black and white movies 
or TV. In his book An Anthropologist on 
Mars, Oliver Sacks described the case of 
an artist (Jonathan I.) who suffered from 
achromatopsia after an automobile acci- 
dent.! This was a catastrophic loss for an 
artist who had previously produced bril- 
liantly colored paintings. He found that 
“now color was gone, not only in percep- 
tion, but in imagination and memory as 
well” There was a “wrongness” to things, 
and photographers will be able to relate to 
the suggested analogy with “Tri-X pushed 
for speed” However, Mr. I. strongly main- 
tained that the concept of gray does not at 
all capture the shades he experiences. 

Color perception, and indeed all vision, 
results from complex multichanneled infor- 
mation processing in the brain. However, 
we must not imagine a unidirectional infor- 
mation flow with data from photoreceptors 
being processed at even more sophisticated 
levels and finally emerging as evolving color 
images for contemplation and reaction. 
Instead there is a high degree of feedback 
at all levels so that what we see depends on 
more than our conscious control of atten- 
tion. Priority is given to what we must see 
for survival. A snake or lion receives our 
immediate attention and initiates action 
before we can consciously consider con- 
sequences. Also, for color to serve as a 
biological marker, there must be some 
constancy in the apparent colors of objects 
even when they are illuminated by quite 


different light sources. This implies a kind 
of automatic white balance adjustment. In 
addition, to some extent we see what we 
expect to see. Fleeting images, especially 
of poorly illuminated objects, often take 
on appearances of well-known things, e.g. 
roots appear to be snakes. 

Furthermore, those who deal with images 
need to recognize that vision with com- 
prehension is a learned process. A normal 
visual system presents an immense array of 
patterns and colors to a newborn baby, but 
initially there is no association of this infor- 
mation with reality. Babies have no perception 
of depth, and small children only slowly 
learn how to put round objects in round 
holes, square things in square holes, etc. The 
child learns to use his or her eyes efficiently, 
to experience depth, to recognize shapes, to 
respond to illusions, etc. It has, in fact, been 
estimated that the complete development 
of vision in humans requires up to 15 years. 
Eventually, a model of reality is learned so that 
an individual is comfortable with their loca- 
tion in space as revealed by the eyes. However, 
adults who have matured in restricted envi- 
ronments, e.g. as members of tribes living 
in tropical rainforests, may still have limited 
depth perception and may fail to recognize 2D 
pictures as representations of the 3D world. 
Here again our understanding of cognition 
and vision has been greatly extended by case 
studies ofindividuals who have regained sight, 
after extended periods of blindness, through 
cataract operations.? A newly sighted individ- 
ual has to learn to see, and the transition from 
blindness to sight is fraught with difficulties. 
There even appears to be mental conflict in 
adapting a brain organized for blindness, with 
enhanced touch and hearing, to a brain organ- 
ized for seeing. The starting point for a baby 
is quite different. Learning may still require an 
extended period, but with a baby there is little 
internal conflict. 
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I want to emphasize that each of us has 
a unique visual system both in capabilities 
and conditioning. We may achieve similar 
scores on visual acuity tests, but still per- 
ceive quite different worlds. Even those who 
show no visual dysfunction may not see sim- 
ilar colors. Language, of course, is no help. 
We all use the same name for the color of 
an object, but no one can know what others 
see. This brings us to a dichotomy that can 
serve as the theme for the study of visual 
perception and photography. Differences 
in perception do not pose a severe problem 
for the design of color cameras, monitors, 
printers, etc., because the goal is to repli- 
cate in an image the optical properties of 
objects in a framed scene. What the eyes 
and brain do with light rays from an object 
or its photographic image is outside the 
realm of color management in photogra- 
phy. As we shall see later in this chapter, 
the theory of light mixing in photography, 
trichromism (George Palmer, 1777; Thomas 
Young, 1802), depends on viewing a small 
visual field with a dark background (the 
void mode). The aim of color management 
in photography is for each point (pixel) in 
the image to reproduce the visual character- 
istics of a point on an object. To go beyond 
this and to control appearance as well, one 
must be able to specify viewing conditions. 

In fact, our perception of colors in a 
scene and ina color photograph of the scene 
depends on the neighboring colors and their 
brightness. Optical illusions also depend on 
image elements in context. While composi- 
tion and the interactions of image elements 
are irrelevant for color management in pho- 
tographic instruments, they are of major 
importance for photographers and other 
artists. To handle this more general require- 
ment, trichromacy is not sufficient and a 
theory of color vision is required that is con- 
sistent with the wiring of the brain. This will 


lead us to the color opponency theory (Ewald 
Hering, 1872), which can also account for 
nonspectral colors such as brown and the 
inhibitory aspects of blue-yellow, red- 
green (cyan), and white-black color pairs. 
Trichromacy and opponency color theories 
were once thought to be contradictory, but 
the proper view is that they are comple- 
mentary, and indeed both are necessary for 
a complete understanding of color vision. 
In fact, electrophysiology and functional 
imaging studies support the idea that tri- 
chromacy describes the detection of light 
by cones in the retina; but that the wiring of 
nerve cells in the retina and the thalamus is 
consistent with Hering’s opponency theory. 
This leaves much to understand. Research 
on the visual system is currently concerned 
with the mechanism of automatic white 
balance (color constancy), the locus in the 
brain of regions that determine color, form, 
position attributes, and even the reaction to 
figurative and abstract art. 


15.2 Color Theories 


Newton and Young-Helmholtz: Modern 
theories of color begin with Isaac Newton 
(1642—1726).* Imagine the young Newton 
setting up a lens and a prism in his dark- 
ened room at Trinity College, Cambridge, 
in 1666. A ray of light from the window, 
defined by an aperture, is focused onto the 
prism. On the output side on the prism the 
ray is spread into a strip of light on a screen 
that glows with the colors of the rainbow. 
This experiment was not completely new. 
Others had seen colored “fringes” from 
lenses and prisms and had presented vari- 
ous explanations. Descartes, for example, 
had concluded that the colors are produced 
by “modifications of white light.” Newton, 
on the other hand, was impressed by the 
fact that the “spectrum” was much longer 
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FIGURE 15.2. Newton's sketch of his optics experiment. 


than it was wide even though the incident 
beam had a circular cross-section. To him 
the experiment demonstrated the separa- 
tion of light into its components revealing 
that “the light of the Sun consists of rays 
differently refrangible” He followed up this 
insight by directing light from a narrow 
(monochromatic) portion of the spectrum 
into a second prism to demonstrate that a 
single “color” cannot be further resolved 
(Figure 15.2). In addition, he used a lens to 
refocus the complete spectrum into a spot 
so that white light was reconstituted. In 


modern terminology, we say that the com- 
ponents of light are refracted by a prism 
each according to its wavelength. 

Also of note is Newton' construction 
of a color circle (Figure 15.3) in which the 
colors are represented by arcs with lengths 
proportional to their extent in the spectrum 
and with red adjacent to violet to connect 
the ends. This was the first of many geomet- 
ric constructions designed to permit the 
estimation of colors obtained by the mix- 
ing of light. Newton did not accept the wave 
theory of light and did not know on what 


FIGURE 15.3. Newton’s color wheel. 
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basis the rays were separated by the prism. 
He advocated a scientific style that relies 
on experiments and mathematics to dem- 
onstrate the truths of nature rather than 
beginning with hypotheses. However, he 
was not immune to preconceived ideas in 
the interpretation of experiments. Consider, 
for example, his assertion that the solar 
spectrum contains seven pure colors. This 
idea, which was to cause some grief to later 
investigators, was motivated by his view 
that the colors in the solar spectrum cor- 
respond to the seven notes in the diatonic 
musical scale.* 

Here is one of many episodes that illustrate 
Newton’s careful experimentation and the 
relevance of his work to modern technology. 
In Book One, Part II, of Opticks, he reports, 
“For I could never yet by the mixing of two 
primary Colors (of light) produce a perfect 
white.Whether it may be compounded of 
a mixture of three at equal distances in the 
circumference (of the color circle) I do not 
know, but of four or five I do not question 
that it may.” Much of the later history of color 
management concerns tricolor matching of 
perceived colors, and in the 21st century 
there has been much interest in design of 
white LEDs (light emitting diodes) based on 
monochromatic LEDs. It has been concluded 
that at least two complementary light sources 
with the proper power ratios are necessary 
for the production of white light, but that a 
“fuller” (easier and more robust) white can be 
obtained by adding more colors. Actually, the 
first commercially available white LED used 
a blue emitting diode coated with a yellow 
emitting phosphor. 

The journey from Newton's initial ideas 
about light mixing to modern concepts of 
color management in photography involved 
contributions from some of the best scien- 
tists over a period of 300 years. Briefly the 
path proceeds as follows. The polymath 


Thomas Young (1773-1829) proposed (in 
1802) that color vision depends on only 
three kinds of color-sensitive cells in the 
retina of the eye that respond to red, green, 
and violet light. He reasoned that the eye 
could not have separate receptors for all 
colors and that three would be sufficient 
as illustrated in Figure 15.4(a). This idea 
was not confirmed experimentally until 
1959. Herman von Helmholtz (1821-1894) 
extended Young’s work on tricolor repre- 
sentation of color by constructing triangles 
to describe color mixing. He demonstrated 
that the pigments required for mixing 
paints are not the same as the colors needed 
in light mixing, and he introduced the three 
variables that are still used to characterize 
a color: hue, saturation, and brightness. In 
the same time period, James Clerk Maxwell 
(1831-1879), famous for developing the 
equations of electromagnetic theory that 
bear his name, showed that almost all colors 
can be derived from red, blue, and green 
(light) and introduced a two-dimensional 
color system based on psychophysical 
measurements. Thus Maxwell was the first 
investigator to use test subjects to judge 
how color samples compare with mixtures 
of primary colors. The relative proportions 
of three colors were recorded; and, since 
that time, these numbers have been known 
as “tristimulus values” 

Light mixing, such as blue and yellow 
to produce white, is the consequence of 
the wiring of the brain, while paint mixing 
is simply the physics of light absorption. 
For example, cyan paint on white paper 
blocks the reflected red light while yellow 
paint blocks reflected blue light. If yellow 
and cyan are both applied, red and blue are 
both blocked and we are left with green as 
illustrated in Figure 15.4(b). In the subtrac- 
tive process, the pigments act as filters and 
their effects are multiplied. 
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FIGURE 15.4. (a) Additive color primaries; (b) Subtractive color primaries. 


15.3 Colorimetry 


n this section we are concerned with 
specifying color stimuli, acting on the 
retina of the human eye, on the basis 
of the Young-Helmholtz color theory. A 
stimulus is light that has been reflected or 
transmitted by an object or perhaps has 


originated from a luminous surface. (Each 
point in a scene contributes a stimulus.) 
This light is characterized by a distribution 
of power in the range of wavelengths that 
affects the cones in the retina. For example, 
the spectral power distribution of radiation 
from an object heated to 5000K is shown 
in Figure 15.5. This particular power source 
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FIGURE 15.5. Spectral power distribution for blackbody radiation at 5000K. 
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approximates the power distribution of the 
sun at midday and can serve as a “white 
point” reference. 

The retina is not equipped to character- 
ize stimuli by measuring the intensity at 
each wavelength, but instead responds to 
the stimulation of the three types of cones. 
If the sensitivity of a cone is known at each 
wavelength, the stimulation can be deter- 
mined by multiplying the power at each 
wavelength (perhaps at 10 nm intervals) by 
the sensitivity at that wavelength and adding 
up the results. The procedure is repeated for 
each type of cone, and the three numbers 
obtained are the tristimulus values with 
respect to the particular set of sensitivity 
functions. These three numbers specify the 
stimulus and are correlated with the color 
that may be perceived; however, it is impor- 
tant to realize that two different stimuli may 
give rise to the same three numbers. Such 
sets of stimuli are called metameric pairs. 
This is, in fact, the basis of modern color 
science since the same perceived color can 
be obtained without having to reproduce 
the original stimuli. 

The analysis of color stimuli and the 
determination of when they match is 
the aim of the science of colorimetry. To 


Test lamp 


2° to 10° view 


A Observer 


understand how this works, imagine an 
observer sitting in front of a small screen 
with a test color projected on the left side. 
The right side of the screen is illuminated 
by beams of red, green, and blue light (the 
color primaries) from projectors that are 
controlled by the observer as illustrated in 
Figure 15.6. The observer is instructed to 
vary the amounts (intensities) of the three 
colors on the right side until the two sides 
match in hue and saturation. When a match 
is obtained, the intensities or values of the 
primaries are noted and another test color 
is projected. Since the visual field is only 
2° to 10° wide and the surround is dark, 
monochromatic test colors can be associ- 
ated with wavelengths. In this way a table is 
constructed that contains the three tristim- 
ulus values for each wavelength of the color 
sample (typically about 380 nm to 740 nm). 
The resulting distributions of values versus 
wavelength are called color matching func- 
tions for the particular set of color primaries 
and the observer. 

For example, Figure 15.7 shows the aver- 
age color matching functions obtained with 
the CIE RGB primaries defined by Red (700 
nm), Green (546.1 nm) and Blue (435.8 nm) 
for a group of observers with normal vision. 


Projectors for 
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FIGURE 15.6. The classic color matching experiment. 
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FIGURE 15.7. Matching functions obtained with CIE RGB primary colors. 


At the wavelength J, this result can be 
described by the equation, S = rR + gG + bB, 
where 7 g, and b are numbers plotted verti- 
cally in the figure; and R, G, and B represent 
the primaries. This equation seems to show 
that any stimulus is equivalent to a sum of 
the proper amounts of three primaries, or 
that any color can be obtained by mixing 
three primary colors. However, the figure 
reveals a region of negative values, and this 
result is found with any set of independent 
monochromatic primaries. The only way to 
match the test color in the “negative” region 
is to add some of the red primary on the test 
color (left) side so that the resulting combi- 
nation can be matched with a combination 
of G and B. This situation is described by 
the equation, S + rR = gG + bB; and by rear- 
ranging we find that S = gG + bB — rR, thus 
justifying the negative values in the figure. 
This brings us to CIE Colorimetry. In 
1931 the Commission Internationale de 
lÉclairage (CIE) met and adopted a par- 
ticular set of color matching functions to 
define the Standard Colormetic Observer. 


At this conference strong personal opin- 
ions and the limitations of technology 
were accommodated by reasonable com- 
promises. Somewhat different decisions 
would be made now, but the standard that 
was adopted served us well into the 21st 
century. The set of CIE primaries shown in 
Figure 15.7 were adopted, but a system was 
established that would provide positive tris- 
timulus values. The idea was to represent 
the characteristics of human observers with 
normal vision; but the choice was, in fact, 
somewhat arbitrary since all sets of color 
matching functions for a given observer 
are simply related to each other through 
mathematical transforms. With any set of 
monochromatic (single frequency) prima- 
ries it had been found that at least one of 
the matching functions must be negative at 
some wavelengths, as shown in Figure 15.7, 
thus a negative tristimulus value would 
appear. Therefore, three narrow band colors 
cannot be combined to match all colors that 
humans can perceive; and, in fact, cannot 
really be primaries. 
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FIGURE 15.8. CIE (1931) color matching functions for the standard observer. 


It is clear that true primaries must 
extend over some range of wavelengths to 
selectively excite the cones in the human 
eye. For mathematical convenience a set 
of “imaginary” primaries was adopted that 
exhibit the required properties. That does 
not mean imaginary in the mathematical 
sense of the word, but, rather, hypotheti- 
cal illuminants that can have negative 
power in some wavelength ranges. These 
hypothetical primaries have the property 
that they can excite a single type of cone 
in the human eye even though the spectral 
ranges of the cones overlap. Therefore, even 
though they cannot be realized in the physi- 
cal world, they can provide reference points 
on the chromaticity diagram and thus serve 
as the basis of the CIE (1931) color match- 
ing functions that are shown in Figure 
15.8. 

The matching functions can then be 
used to compute the CIE tristimulus values 
X, Y, and Z for any stimulus S(4) by means 
of the following summations: 


700 
X=k SA) 20) 
A=380 
700 
Y=k SA) yA) 
A=380 
700 
Z=k SA) ZA) 
A=380 


(15.1) 


(When k is chosen so that Y = 100 for a 
perfectly white object, the Y values are called 
percent luminance values.) Equation (15.1) 
is simply a precise, shorthand description of 
the procedure outlined above for determin- 
ing the effect of a stimulus on each type of 
cone. Basically, the stimulus values at evenly 
spaced wavelengths are multiplied by the 
sensitivities of the cones at the same wave- 
lengths and the results are added (summed). 
When the light is reflected from an object, 
it is appropriate to express the stimulus 
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S(A) as the product of the power distribu- 
tion of the light source, /(1), and the spectral 
reflectance of the object, R(A). The following 
tristimulus value ratios, also known as chro- 
maticity coordinates, are defined so that 
chromaticity can be plotted in two dimen- 
sions: x = X/X, y = Y/X, and z = Z/X, where 
L=X+Y+Z.Sincex+y+z=1,all of the 
coordinates can be represented by plotting 
y versus x as illustrated in Figure 15.9. From 
the definitions, all computed chromaticity 
coordinates are positive, and white is rep- 
resented by x = y = z = 1/3. 

This diagram should be carefully studied. 
Light sources that are defined by a single 
wavelength, i.e. monochromatic light, give 
stimuli that fall on the “locus of the spec- 
trum” line. The area defined by the spectrum 
and the “purple boundary” contains the 
coordinates of all stimuli that can be seen by 


the CIE standard observer. Furthermore, all 
stimuli that can be obtained by mixing the 
CIE RGB primaries fall inside the dotted tri- 
angle. This triangle represents the “gamut” 
of the CIE RGB color space. 

Actually, gamut refers to the subset of 
colors limited by a set of primaries, includ- 
ing the range of brightness levels from the 
darkest black to the brightest white. The 
RGB primaries are “real” since they can be 
realized in the physical world; but the CIE 
color space is based on the imaginary pri- 
maries located at Red (x = 1, y = 0), Green (x 
= 0, y = 1) and Blue (x = 0, y = 0). What this 
means is that, in principle, the “imaginary” 
red primary, if substituted into Equation 
(15.1), would give x = 1, y = 0, and so on. 
(For additional discussion, see the book by 
Giorgianni and Madden.”) It is clear from 
Figure 15.9 that the imaginary primaries 


FIGURE 15.9. The CIE chromaticity diagram with the CIE RGB color space. 
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must be chosen so that their gamut includes 
all stimuli that can be seen by the standard 
observer and that the resulting tristimulus 
values can be positive. However, the triangle 
defined by x = 1, y = 1, and z = 1 necessarily 
encloses sets of tristimulus values (outside 
the spectral boundary) that can neither be 
produced nor observed. 

Figure 15.9 also shows white reference 
points near the center. The blue circles label 
the chromaticity coordinates for blackbody 
radiation computed in 500K steps. The 
5000K (D50) circle was computed by using 
the power distribution shown in Figure 15.5 
and the CIE matching functions (Figure 
15.8) in Equation (15.1). Coordinates 
between the 5000K white point and a point 
on the spectrum curve represent varying 
degrees of saturation ranging from 100% on 
the spectrum line to 0% at the white point. 

Note: The standard illuminant D50 is 
constructed to represent natural sunlight.* 
The same chromaticity coordinates can also 
represent light sources that have spectral 
distribution functions that are far from that 
of sunlight. Hence, 5000K on a fluorescent 
label may not mean the same thing as D50. 
This is not a trivial difference, since absorp- 
tion, reflectivity, and polarization can 
depend on wavelength. For example, your 
jacket may not appear to be the same color 
when viewed under D50 and 5000K light. 

It is common for the chromaticity dia- 
gram to be presented in color with white 
near the center, then pastels, and finally sat- 
urated colors in the spectrum at the edges. 
This can be beautiful, but it is a fantasy. The 
CIE color system can represent a stimu- 
lus but not a perception. It can be used to 
specify differences between stimuli but not 
the appearance of a stimulus. As previously 
described, the appearance depends on the 
surrounding colors and the viewing condi- 
tions. The same stimulus, in fact, may give 


the appearance of quite different colors 
under different conditions. 

Two points need to be made before I leave 
this general discussion of color representa- 
tion. First, the gamut of accessible colors for 
a color space depends also on brightness 
and, therefore, must be represented in three 
dimensions. The xy plot is, of course, only 
a projection. The 3D representation shows 
limitations on our ability to obtain bright- 
ness for certain colors. For example, highly 
saturated colors, other than yellow, are only 
obtained with low luminance. 3D represen- 
tations of various color spaces are available 
online. They are nice to look at, but usually 
not very useful for photographers. As with 
the xy plots, there is an irresistible tempta- 
tion to decorate them with colors. Also, the 
perception of size or volume of color spaces 
in 3D can be misleading. 

This leads me to the second point. The 
human eye is not equally sensitive to shifts in 
the different directions in the chromaticity 
diagram. Ifa particular stimulus is selected, 
i.e. a point in xy space, observers can more 
accurately match that stimulus with combi- 
nations of primaries that vary in a particular 
direction than in combinations that are 
perpendicular to that direction. Economic 
interests have driven a lot of research in this 
area because of the desire to provide the 
best representation of color at the lowest 
cost. Some of the conclusions are summa- 
rized in MacAdam ellipses, constructed at 
various points in the chromaticity diagram, 
that encompass all the colors that are indis- 
tinguishable to the human eye. Roughly 
speaking, in the region where y is greater 
than 0.5 the ellipses tend to be long in the 
direction toward the peak of the spectral 
horseshoe (520 nm). One MacAdam ellipse 
is illustrated in the upper part of Figure 
15.9.? This emphasizes the need for color 
spaces that have perceptional uniformity 
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so that the distance between points is cor- 
related with differences in perceived color. 
The CIE LAB and CIE LUV color spaces 
are attempts at nonlinear compressions of 
the CIE XYZ to improve uniformity. These 
are 3D spaces in which the “lightness” 
scale is proportional to the cube root of 
the luminance as an approximation to the 
logarithmic response of the human eye to 
luminance. The CIE LAB color model is 
now widely used and is available in standard 
imaging software. 


15.4 Color Spaces for Digital 
Photography 


ith the almost universal accept- 
ance of digital photography 
along with computer processing 
of images, photographers face the con- 
cept of color management daily whether 
they recognize it or not. We all want our 
images to have the same appearance on 
the camera LCD, the computer screen, in 


prints, and perhaps in projection as well. 
This may be taken for granted until obvi- 
ous and distracting color shifts are noticed, 
and then we want to fix it. The task is 
complicated by the fact that each presen- 
tation device has a native color space and 
gamut. The first step in the color pathway 
is often a digital camera fitted with a sen- 
sor that captures the image through three 
color filters, usually red, green, and blue. 
Each filter covers about one third of the vis- 
ible region and there is some overlap of the 
regions. 

The detailed specifications of sensors 
are proprietary and are often not available 
to the public. Fortunately, Kodak offers data 
sheets online for their catalog of sensors. 
For example, the spectral response of the 
sensor in the Leica M8 is shown in Figure 
15.10. A sensor behind a red filter, for exam- 
ple, will accumulate a number of electrons 
in response to a stimulus, and this number 
will be digitized and processed off-sensor 
to represent red. The user does not know 
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FIGURE 15.10. Spectral response of the Kodak sensor used in the Leica M8. Used with permis- 


sion of Eastman Kodak Company. 
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for sure even what processing and “error” 
correction goes into constructing the RAW 
image. 

So what color space should be used 
in an image processing program such as 
Photoshop? The Leica M8 sensor obvi- 
ously detects the entire spectrum visible to 
humans and in principle has a gamut greater 
than the human eye. Why not use a camera- 
defined color space? That is a possibility, but 
experts in color management agree that 
standard color spaces such as sRGB, Adobe 
RGB, etc. offer advantages. For those who 
select JPG file format, or have it selected for 
them, the choice of color space (usually sRGB 
or Adobe RGB) must be made in the camera 
since it is burned into the file. Full-featured 
cameras offer the choice of a RAW file for- 
mat which leaves the specific camera based 
color space in place until the conversion 


step. At that point the user must assess 
the advantages of the various alternatives. 
If the color space is too small, informa- 
tion will be lost and up-conversion cannot 
recover the information. So it is important 
to choose the color space with the larg- 
est practical gamut initially, realizing that 
down conversion to match the gamut of the 
web or a projector can easily be made later 
without loss. 

Some popular color spaces are illus- 
trated in Figure 15.11. The sRGB space was 
designed to make use of the properties of 
typical monitors and printers in the late 
1990s, but with the advent of large gamut 
LCD monitors and printers with eight or 
more colors of ink a wider gamut is advisable. 
The Adobe RGB (1998) has been available 
for several years and is a good alternative 
to sRGB, at least for image processing. But 
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FIGURE 15.11. Common color spaces plotted on the CIE chromaticity diagram. 
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now even Adobe RGB is being challenged. 
Some professional photographers advocate 
the use of the Reference Output Media 
Metric (ROMM) or ProPhoto color space. A 
glance at Figure 15.11 shows that ProPhoto 
makes use of imaginary primaries and 
characterizes some stimuli that cannot be 
displayed are even observed by the human 
eye. 

So why not use the largest possible color 
space just to be on the safe side? There are 
two problems. First, the range of colors 
that must be covered by a certain number 
of digitized steps is larger even though part 
of the range is useless. This can lead to 
obvious steps in shades of colors when 
8-bit digitization is used. With 14-bit or 
16-bit processing this is unlikely to present 
a problem. The second concern is that more 
information must be stored. The conclu- 
sion is that one must decide what is large 
enough. Professionals and serious amateurs 
who expect to make large prints with print- 
ers that use six or more colors of inks or 
pigments may wish to select the ProPhoto 
color space now and to use special proof- 
ing techniques to ensure that the additional 
colors can be displayed or at least handled 
in a reasonable way. If photographs pro- 
cessed in the wider gamut color spaces need 
to be displayed on the web or projected, it 
will usually be necessary to convert them to 
the sRGB space. 


15.5 Color Management in the 
Tristimulus World 


he point of color management 
is to maintain the appearance of 
colors in images from the capture 
(digital camera or scanner) through the 
display (monitor) and onto the printout 
(inkjet, laser printer, or press profile). This 
involves an extensive area of convention 


and technology that is the subject of several 
monographs. Here is a brief summary of 
what is involved. Each device is character- 
ized by a profile that specifies at least the 
gamut (characteristics of the colorants) and 
the dynamic range (white and black points). 
The image processing system in a computer 
acts as command central. It receives data 
with input profiles and provides a profile 
connected space (PCS) selected by the user. 
In other words, the RGB values from the 
source are converted to coordinates for the 
PCS. Another conversion is required to pro- 
vide RGB values to the monitor according 
to the monitor profile. This assumes that the 
monitor has been calibrated and a profile is 
available. 

Color reproduction in the monitor 
depends on the gamut of the monitor, and 
that is likely to be smaller than the gamut 
of the color space of the PCS. For exam- 
ple, the monitor may be able to display all 
colors in the sRGB space but not the com- 
plete ProPhoto space. This leads to the little 
understood topic of rendering intents. That 
is, if you need to convert to a smaller color 
space, how do you clip colors? Do you throw 
away the out-of-gamut colors (relative col- 
orimetric) or attempt to preserve some 
relationship between out-of-gamut colors 
even at the expense of in-gamut colors (per- 
ceptual)? These are not the only choices, and 
this is not just an academic exercise, since 
choices must be made at least when print- 
ing parameters are selected. Furthermore, 
the printer will have a different gamut from 
that of the monitor, and how can we 
know how the printed image will appear? 
Fortunately, image processing programs 
provide for previewing the printed image on 
a monitor and at detecting the out-of-gamut 
colors. For additional information, the book 
Color Management, by Fraser, Murphy, and 
Bunting, is recommended.” 
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Instruments are available that make color 
management relatively easy for photogra- 
phers. Full calibration should encompass 
the camera, monitor, printer, and projec- 
tor; however, the most important step for 
most photographers is monitor calibration. 
X-Rite, Datacolor, and a few other compa- 
nies offer monitor and printer calibration 
devices. These attach to the monitor screen 
and detect the colors displayed for a series 
of sets of tristimulous values. A profile is 
then prepared for the monitor so that that 
the computer can correlate the monitor 
input signals with the colors actually pro- 
duced. The same idea is used to calibrate 
printers and projectors. For example, cali- 
bration of a printer with a certain type of 
paper requires that a set of color patch 
prints be made and then scanned into the 
computer for analysis. This usually requires 
that the procedure be repeated more than 
one time to achieve a satisfactory profile. 
Fortunately, paper manufacturers often 
provide free profiles for their papers that are 
adequate for most users. Projector calibra- 
tion is also very important. This calibration 
can be accomplished with the Colormunkie 
device from X-Rite that monitors the pro- 
jector screen and produces a profile in the 
same way that the monitor profile was cre- 
ated. In a camera club setting with a shared 
projector, of course, a profile will have to be 
created with X-Rite software for every com- 
puter that is used with the projector. Other 
projector calibration systems are offered 
by X-Rite, Datacolor, and Integrated Color 
Corporation. 

I have left the camera color profile until 
last because the choices of profiles built into 
Photoshop and Lightroom and probably 
other software as well are usually adequate. 
More demanding users can create custom 
profiles with color charts such as those in 
the X-Rite ColorChecker Passport and 


Datacolor SpyderCHECKER. The software 
supplied with these products works auto- 
matically with selected photoprocessing 
software packages. X-Rite creates Adobe 
DNG profiles and does not support ICC 
profiles. The procedure is to photograph 
a color chart under lighting conditions of 
interest, such as sunny day or cloudy day. 
As an example, consider profile creation in 
Lightroom with the X-Rite Passport plug- 
in. The RAW image requires no editing and 
automatically produces a camera profile. 
One simply selects the export option with 
X-Rite Presets. If the image is properly 
exposed, the result will be a camera-specific 
profile that can be named for identification 
of lighting conditions. The profile permits 
software correction of colors recorded 
by the camera to match the true tri- 
stimulus color values of the scene that was 
photographed. 


15.6 Goethe-Hering Theory 


he success of trichromacy, or Young- 

Helmholtz theory, does not mean 

there has been no controversy. In 
fact, there is an alternative color theory 
that has its origins in the work of Johann 
Wolfgang von Goethe (1749-1832). Goethe 
is justly famous for his contributions to art, 
literature, drama, and even science; his best- 
known work is the dramatic poem Faust. 
Therefore, it is surprising that he wanted to 
be remembered for his study of the percep- 
tion of color. From a few observations with 
a prism, borrowed from a friend, in 1790 
Goethe became convinced that Newton 
was wrong about composition of color. 
This led to many years of experimentation 
and heated polemic. Goethe misinterpreted 
his prism experiments and apparently mis- 
understood Newton’s explanations. He 
favored the Aristotelian theory that colors 
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result from the interplay of dark and light, 
and there were many other observations 
that convinced him that Newton had missed 
important points. Chief among these were 
the apparent colors of shadows and after- 
images of color. Famously, he described 
staring at a comely girl wearing red in a bar 
and then seeing a green “after image” when 
he looked away. He concluded that there are 
pairs of colors that stand in opposition to 
each other and cannot be mixed. 

Scientists and many others were repelled 
by Goethe’s attacks on Newton, then dead 
100 years, and his promotion of “unsci- 
entific” ideas.!*!>1* He failed to discredit 
Newton, and he did not provide a satisfac- 
tory alternative theory. However, he was on 
to something that Newton missed. Newton 
dealt with the physics of light and did not 
take into account the slippery relationship 
between subjective colors and “refrangea- 
bility” of rays. His writings do indicate that 
he mused about the problem of how the 
“Phantasms of Colors” are produced. In fact, 
so much of what we perceive through vision 
is determined by physiological factors that 
many knowledgeable neurologists and even 
physicists now believe that “Goethe got it 
right about color” That, of course, does not 
detract from what Newton got right about 
light. 

In fact, the rehabilitation of Goethe has 
resulted mainly from the relation between 
his ideas and the successful Hering theory 
of color opponency. Ewald Hering (1834— 
1918) provided another explanation for 
color mixing that appeared to be very dif- 
ferent from the Young-Helmholtz model. 
From visual experience, he recognized 
that red, green, yellow, and blue have spe- 
cial standing. In place of the three primary 
colors (trichromacy), Hering envisioned 
(in the eye and/or brain) three opponent 
processes. He proposed that the pairs 


blue-yellow, red-green, and dark-light con- 
trol our perception of color. In this way he 
could explain the strange properties of light 
mixing that had been discussed at least 
since the time of Goethe. First, yellow and 
blue do not mix, and red and green do not 
mix. The statement that blue and yellow 
make green is incorrect for light. The fact 
that some people believe it probably results 
from their experience with the mixing of 
paint or pigment, which is a subtractive 
(absorption) process. The addition of blue 
and yellow light, in fact, gives white light 
if the proper hues and intensities are used. 
Similarly, the addition of red and green light 
gives yellow. There are no bluish yellow or 
reddish green colors. Actually red and cyan 
give white, so that is probably a better oppo- 
nent pair. 

Photographers will recall the color bal- 
ancing tools in programs such as Photoshop 
where the sliders range from yellow to blue, 
cyan to red, and magenta to green. These 
are antagonistic processes where one color 
appears at the expense of the other. White 
and black, of course, depend on contrast, so 
that black and gray are experienced because 
they are less bright than the surround- 
ing area. Hering was able to explain the 
appearance of nonspectral colors by oppos- 
ing effects of light and dark. For example, 
brown cannot be matched in the classical 
color-matching experiment. It only results 
from our perception of yellow or orange 
surrounded by brighter light. 

The Hering theory had many successes 
and it won over many psychologists and 
others. However, physical scientists had 
problems with the theory because it did 
not seem to have any physical basis. After 
all, there are three primary colors and tri- 
chromacy works in most cases. We now 
know that the Young-Helmholtz theory 
describes the operation of photoreceptors, 
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and the Hering theory is related to the oper- 
ation of the visual system at later stages in 
the retina and the brain. Hering was indeed 
prescient. His insight, or intuition, led him 
to an amazingly good description of the 
operation of the opponent visual system 50 
years before it could be verified by experi- 
ments. This is reminiscent of Young’s insight 
or guess that there were three types of color 
receptors in the eye. 


15.7 Monochrome and 
Monotone Images 


here is some confusion about what 
Sk a monochrome image. 

This confusion may arise because 
terminology in photography differs from 
that in traditional art. From color theory 
and photographic usage, it is clear that mon- 
ochrome means black and white. In terms 
of the RGB color space, every pixel in the 
image must exhibit three identical values 
for R, G, and B. For example the RGB val- 
ues ofa white region might be (255,255,255) 
and those for a black region will be close to 
(0,0,0). At some intermediate value in the 
gray scale the result might be (50,50,50). 
On the chromaticity diagram (Figure 15.9) 
only a single point is represented, and that 
point is near the center of the diagram at 
a particular white point (depending on the 
color temperature) denoted by the chroma- 
ticity coordinates x and y. This can easily 
be verified with standard photo processing 
programs. In Photoshop, gray scale can be 
selected in the mode menu or monochrome 
can be selected in the Channel Mixer win- 
dow. The resulting image can be reset to 
RGB and examined in the Info palette to 
determine the RGB values for any pixel. 
This is very different from desaturating 
two colors or manipulating images in the 
Channel Mixer, procedures that only lead 


to different ratios of RGB values in differ- 
ent pixels and another color image, albeit 
different. 

So is it possible to obtain a monochrome 
image that has a color or hue? This would 
mean an image in which the RGB values for 
every pixel have the same ratios of R to G to 
B values. That is to say every pixel would be 
represented by the same point in the chro- 
maticity diagram and would have the same 
hue and saturation. If the primary color 
blue was chosen, the values would range 
from (0,0,0) to (0,0,255) for 8-bit color and 
all pixels would have 0,0 for the R,G values. 
This would not be very interesting as there 
would be no shades of blue; and, in fact, 
pixels would range from bright saturated 
blue to very dark saturated blue (black). 
Furthermore, there would be no white! This 
kind of image can be viewed (though not 
printed in color) in the Photoshop Channels 
palette by turning off the red and green 
channels and selecting the color display 
option. Of course, some other hue and satu- 
ration could be picked for the monochrome 
image, but white could not appear in the 
image unless gray scale was chosen. For 
that reason, there is essentially no interest 
in single hue monochrome except perhaps 
as a faint tint on gray scale images. 

In contrast to this clear definition, artists 
sometimes refer to a painting as mono- 
chrome when it is painted with shades on a 
single color. The problem here is that mix- 
ing a pure color (hue) with white changes 
the position in the chromaticity diagram. 
Similarly, projecting blue light and white 
light onto a screen produces a pastel shade 
that is, in fact, a new color. White light 
contains equal amounts of R, G, and B and 
the blue light only contains B. Therefore, the 
combined RGB values do not maintain the 
same ratios from pixel to pixel. This can be 
expressed in the hue, saturation, brightness 
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Monotone 


Color Monochrome 


FIGURE 15.12. An image presented in Color, Monochrome, Color Monochrome, and Monotone. 


(hsb) system by noting that the saturation 
has changed. Still another example would 
be an inkjet printer that has only blue ink. 
The ink dots would either be dense on paper 
so that saturated blue would appear or 
sparse so that the white paper and blue dots 
would simulate other shades. In either case 
the shades of blue would be represented 
in the chromaticity diagram on a line con- 
necting the blue hue with the white point. 
What I am describing is a monotone image 
not a monochrome image. A monotone 
image can be produced in Photoshop by 
gray scale conversion followed by selection 
of Duotone. The monochrome image can 
then be converted to monotone by selecting 
one ink. 

The kinds of images discussed here are 
shown in Figure 15.12. If the RGB values 
of these images are examined (for exam- 
ple, in Photoshop) the values for the color 
image will be found to range from (0,0,0) 
at the darkest point to (255,255,255) at the 
brightest point with a wide range of val- 
ues and ratios of values in between. The 
Monochrome image also ranges between 
(0,0,0) and (255,255,255) but every set 
contains three identical numbers. The 
Color Monochrome should have RGB val- 
ues that range from bright blue (0,0,255) 


to near black (0,0,0) with R and G values 
always zero. Obviously, there is no white 
in this image. In interesting contrast, the 
Monotone image ranges from (255,255,255) 
to (0,0,255) with the R and G values always 
equal. Therefore, white is present but there 
is no black! In conclusion, for all practical 
purposes, monochrome means black and 
white. Monotone (one ink) and duotone 
(two inks) images may be of interest for 
special purposes, such as advertising illus- 
trations. See, for example, the monotone 
package from Twisting Pixels. 


15.8 Color Constancy and 
the Land Theory 


dwin H. Land, the founder of Polaroid 
E Corporation, reported a set of exper- 

iments in 1958 that amazed even 
the experts in color theory.!* In a research 
project that was associated with the devel- 
opment of Polaroid color film, Land and 
his co-workers were repeating the classic 
color photography experiment of Maxwell. 
This required them to take three images of 
a scene on black and white film using red, 
green, and blue filters. From these images, 
slides were made that could be projected 
in register on a screen. The image acquired 
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with a red filter was projected through a red 
filter, the blue one through a blue filter, and 
the green one through a green filter. This, of 
course, worked; and a brilliant color image 
was obtained. The principles are known 
and form the basis of commercial three 
color projectors, but an unexpected thing 
happened at closing time. The blue projec- 
tor was shut down and the green filter was 
removed from the green projector, leaving 
only a black and white image (from the slide 
taken with a green filter) and the red image. 
Before shutting down the red projector, 
Land noticed that the screen still showed 
a full color image instead of the expected 
pink image. At the time he thought it might 
have resulted from some kind of adapta- 
tion process in the eye, but later he realized 
that something remarkable was happening. 
This prompted him to undertake decades of 
research on the perception of color and ways 
to model the operation of the visual cortex. 
In later experiments a Mondrian-like 
patchwork of colored papers was used to 
test color perception under a variety of illu- 
mination conditions. The major finding was 
that a given color patch, say the green one, 
maintains its green appearance in the con- 
text of the surrounding patches even when 
the illuminating light sources are changed so 
that the stimulus from that patch equals the 
stimulus that previously had been measured 
for a blue or a red patch. It turns out that 
color contrast across borders is all impor- 
tant for color vision. In his 1983 follow-up, 
Land emphasized his conclusion, “whereas 
in color-mixing theory the wavelengths of 
the stimuli and the energy content at each 
wavelength are significant in determining 
the sense of color... . in images neither the 
wavelength of the stimulus nor the energy at 
each wavelength determines the color. This 
departure from what we expect on the basis 
of colorimetry is not a small effect, but is 


complete” In the Retinex theory proposed 
by Land, there are three mechanisms at 
work to determine the lightness of areas at 
three different wavelengths. The important 
fact in color vision is that an image is simply 
an array of areas with different lightnesses 
relative to each other. The computational 
challenge in Retinex theory is to show how 
color can be predicted from the lightness of 
the various areas without any information 
about the illuminant. 


15.9 Color Opponent Cells in 
the Retina and Brain 


arious aspects of our visual expe- 

rience are captured by the color 

theories described above, but we 
need to discover the underlying physical 
basis for these effects. The lucky “guesses” 
by Young and Hering brought us the 
immensely important ideas of three color 
receptors and opposing colors, respectively, 
long before there was physical evidence for 
mechanisms behind them. But in the last 
few decades, brain research has advanced 
to the point that we now have physical 
models for the visual system that can rec- 
oncile trichromism and color opponency. 
It is clear that our visual system operates 
by the transfer and processing of informa- 
tion about images. The story proceeds as 
follows. The cones, S, M, and L (for short, 
medium, and long wavelength, respectively) 
are connected through bipolar cells to gan- 
glion cells in such a way that the ganglion 
cells respond to sums, differences, and per- 
haps other combinations of cone signals. In 
this way arrays of ganglion cells with differ- 
ent types of receptor fields are formed. 

The luminance cell, which responds to 
the sum S+M+L, is arranged with a center 
that responds to a light signal and a sur- 
round field that suppresses the response 
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On center 


Off center 


FIGURE 15.13. Luminance (ganglion) cells in the retina. 


(Figure 15.13), or the reverse where the 
on-center signal suppresses the signal and 
the surround field responds. In the classic 
experiment of Hubel and Wiesel, a spot of 
light was found to increase (“on cell”) or 
decrease (“off cell”) the firing of a nerve fiber. 
In each case when the size of the spot was 
increased, the response was reduced by the 
action of the surround (“off center”) cells as 
illustrated in Figure 15.13. The size of the 
center field depended on the position on 
the retina, but was typically about 3° of arc. 

The opponent aspect here is spatial, and 
black and white are not opposite colors. 
We can, of course, experience any shade 
of gray. The striking visual effect exhibited 
by the grid in Figure 15.14 has been used 
as an illustration of the suppression of the 


center by the surround field. However, the 
situation may not be so simple, and this 
explanation is now controversial. 

Things get even more interesting with 
color. Here the wiring of the nerve field 
transforms from RGB cone signals to 
opposing signals on the axes shown in 
Figure 15.15. For example, we find cells 
that can be excited with red light (L) and 
suppressed with green (cyan) light (M) or 
vice versa. These simple R+C- and R-C+ 
opponent cells have coexisting L and M 
cones, and we find similar arrangements 
of L+M (yellow) versus S (blue). In Figure 
15.15 a plus sign indicates excitation and 
a minus sign means suppression. A cell 
either transmits red or cyan but not both, 
and, similarly, a cell transmits either blue 


FIGURE 15.14. Elke Lingelbach’s version of the Hermann grid illusion (1870). 
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FIGURE 15.15. Single opponent cells. 


FIGURE 15.16. The color opponent axes against 
the CIE diagram. 


or yellow, not both. The relation of the 
opponent colors to the CIE chromatcity 
diagram is shown in Figure 15.16. 
Transmission of information from the 
retina is in opponency form, and later pro- 
cessing in the visual cortex makes use of 
more sophisticated opponency cells. For 


example, a pair of double opponency cells 
are shown in Figure 15.17. The red/cyan 
double opponency cells require excitation 
in the center cell by red cones and excitation 
in the off-center (surround) by cyan cones 
to be able to fire a nerve fiber. 

The opponency structure reminds one 
of the Hering ideas about opponency, but 
the variety of simple and double opponency 
cells is much richer than that suggested by 
Hering’s theory. The question naturally 
arises about why nature arranged things 
this way. One suggestion is that the lumi- 
nance system evolved first and that the 
color opponency system was added on top 
of it. Furthermore, the separation of the M 
and L cones from a single common ancestor 
cell was a fairly recent event in the evolution 
of the color system that is only found in old- 
world primates. This resulted in additional 
types of opponent cells and programming 
for information transfer. 

Undoubtedly, the major advantage of 
opponent coding by the luminance cells 
and the double opponency cell in the color 


RED+CYAN- 


RED-CYAN+ 


FIGURE 15.17. Red/cyan double opponency cells. 
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system is the ability to detect “edges” Any 
abrupt spatial change in color or luminance 
will cause an imbalance in the center and off 
center cells and will lead to increased firing 
of nerve fibers. Similarly, areas of uniform 
luminance or color will tend to balance 
excitation and suppression and will reduce 
the rate of nerve firing. In this way colors 
can be established at the boundaries of spa- 
tial regions, and the interior regions can be 
filled in with relatively little information. 
Image compression in modern computers is 
based on similar ideas. If a color is constant 
for some distance in an image, there is no 
reason to record its value at every point. For 
example, blue may extend for 1000 pixels 
in an image, and in each pixel blue has the 
value 255. Instead of recording 1000 values 
of 255, it can just be noted that 255 occurs 
1000 times. 

The Nobel Prize was awarded in 1981 to 
David Hubel and Torsten Wiesel for their 
studies of information processing in the vis- 
ual system, and research continues to reveal 
the features of this amazing system.'”** 
However, this work is just at its beginning 
because the really hard part concerns high 
level processing in the visual cortex and 
beyond. Even our understanding of the parts 
that have been extensively studied is incom- 
plete. For example, the ratio of L to M cones 
in the average eye is about 2:1, but this ratio 
varies by a factor of 4 for different individu- 
als apparently without affecting color vision. 
The S cones only make up about 8% of the 
total and there are essentially no S cones in 
the fovea. Furthermore, the distribution of 
Land M cones in the retina is not uniform. 
One wonders how the receptor fields for the 
ganglion cells can be constituted. 

The brain appears to be performing 
magic when it converts streams of encoded 
information from two eyes into an illusion 
of a 3D full color world. In fact, the work 


recently reviewed by Werblin and Roska” 
identifies two-dimensional arrays of 12 
types of ganglion cells in the retina that 
transmit partial representations of the vis- 
ual field as “movies” to the brain. One image 
stream contains edges information, another 
shadows, another motion and direction, etc. 
This is done in real time with a large amount 
of feedback at each stage. Thus, our reality 
depends on the brain’s interpretation of an 
array of continuously updated partial visual 
representations. 

Vision is the primary way we obtain an 
understanding of the world. It also pro- 
vides great enjoyment and sometimes awe 
and wonder. The introduction to the visual 
system presented here reveals practical 
aspects of color management, and it also 
gives a peek at the array of special effects 
we experience because of the particular 
architecture of the visual system. By this, 
I mean color constancy, color groupings 
that are striking, optical illusions that result 
from opponent cells, etc. In Chapter 18, I 
consider perhaps unanswerable questions 
concerning human creation and apprecia- 
tion of art. Can science help us understand 
why humans are attracted to art and fic- 
tion in general? Artists are perhaps acting 
as neurologists as they experiment with 
the framing of scenes, use of color, repre- 
sentation of light, etc. to determine what is 
pleasing to the human brain. 
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CHAPTER 16 


Image Capture and Processing 


Teams of Bell Labs scientists, such as Shockley, Brattain, Bardeen, 
and many others, met the challenge—and invented the information 


age. They stood on the shoulders of the great inventors of the 


19th century to produce the greatest invention of our time: 
the transistor. 


—WWW.IDEAFINDER.COM/HISTORY/INVENTIONS/TRANSISTOR.HTM 


Humankind eventually would have solved the matter, 
but I had the fortunate experience of being the first person 
with the right idea and the right resources at the 
right time in history. 


—ACK KILBY 


16.1 Introduction 


hotography, or “writing with light,’ 
p: all about capturing images. Until 
the 21st century, that usually meant 
using photographic film and developing it 
to produce negatives or slides. Now only 
old-timers remember or care much about 
film, and it appears to be on the way out. 
Film is mainly of historical interest, but it 
is instructive to compare its characteris- 
tics with those of digital sensors. Also, the 
principles for capturing and reconstructing 
color images still apply. 
The importance difference between 
photographic films and digital sensors 


comes down to how the image is sampled 
and recorded so that it can be reproduced 
with fidelity. The image of concern here 
is a pattern of light projected by a lens onto 
a sheet of film or a digital sensor. Recall 
from Chapter 2 that light consists of pack- 
ets of energy called photons. To be more 
precise, the energy of a photon is given by 
E = hv where v is the frequency of the light 
and » is a fundamental constant of nature 
(h = 6.626 x 10°. s). It is the interaction of 
photons with matter, usually electrons, that 
permits light to be detected and its intensity 
to be measured. This sounds abstract, but it 
is the essential part of understanding how 
photographic images can be recorded. 


168 oy CHAPTER 16 


In order to understand what photons 
do in film photography, it is necessary to 
look at the photochemistry and photo- 
physics at the atomic level. This is indeed 
interesting, but I have relegated it to 
Appendix G so that we proceed immedi- 
ately to the operation of digital sensors. We 
will be concerned with sensitivity, resolu- 
tion, and image processing. There are still 
some areas where photographic film excels, 
but with the rapid advance of digital tech- 
nology, it is clear that the use of film will 
continue to decline. In the 21st century I 
expect that film will be viewed as no more 
useful for photography than vinyl records 
are for audio reproduction. 


16.2 Capturing Photons 
with Image Sensors 


ensors have all but replaced film for 

image capture, but this is just the tip of 

a mountain of advances in science and 
technology. Sensors and their use would be 
impossible without the microelectronics 
industry and the plethora of new products, 
especially microprocessors and memory 
chips. The information revolution, based 
on microelectronics, expands exponen- 
tially, sweeping away last year’s marvels and 
presents seemingly unlimited features and 
opportunities. By 2008, we had tiny mobile 
telephones that could surf the Internet, 
take and transmit photographs and movies, 
display geographic coordinates and maps, 
and on and on. Our cameras are now part 
of the information explosion just as com- 
puters are, and we can expect that they will 
become obsolete every few years. The rate 
of change will eventually slow down as it has 
for other mature technologies such as tel- 
evision and automobiles, but the transition 
period is far from over for cameras. The his- 
tory of microelectronics and the path to the 


digital sensors now available to photogra- 
phers is a fascinating story. It is summarized 
in Appendix H. 


16.3 CCD and CMOS Image Sensors 


oth CCD and CMOS sensors are used 

in digital cameras, but most users are 

not aware of their differences.!” Their 
names indicate their construction, which is 
not of direct interest here. This chapter will 
focus on the way sensors work and their 
relative advantages, at least at this stage of 
their development. In both CCD and CMOS 
sensors the pixel array can be thought of as 
an array of small buckets designed to catch 
photons. These buckets are usually square 
in cross-section, but may have a variety 
of shapes. For example, Fujifilm has used 
octagonal pixels in their Super CCD sensors 
and Nikon tried rectangular pixels in their 
D1X DSLR (see Figure 16.1). In interchange- 
able lens cameras, the pixel pitch (edge 
dimension) is typically 5-8 um, in point- 
and-shoot cameras it may be less than 2 um, 
and in cell phone cameras it may approach 
1 um. A photon detector occupies most of 
the surface area of each pixel in a CCD sen- 
sor, while in CMOS sensors the detector 
must share the pixel with other circuit ele- 
ments. Therefore, there is an advantage to 
the CCD design that is more significant for 
very small pixels. However, the advantage is 
shrinking with time because circuit elements 
are becoming smaller, allowing more efficient 
use of space in a pixel, and pixel-covering 
micro lenses are being used to focus inci- 
dent light onto the detector. Furthermore, 
back-illuminated CMOS sensors can greatly 
increase the efficiency of photon capture. In 
this design the layer of detectors is moved 
closer to the light-gathering lenses and 
the wiring and electronic components are 
located behind the detectors. 
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| Pixel pitch 


FIGURE 16.1. Pixel arrays: (a) square, (b) octagonal, and (c) rectangular. 


Ina photon detector, an absorbed photon 
produces an electron-hole pair and thereby 
makes a mobile electron available for stor- 
age in a capacitor. The quantum efficiency 
of the detector is defined as y = (number 
of electrons/sec)/ (number of photons/sec). 
This number depends on the frequency 
(color) of the light; but for modern sen- 
sors is typically around 50% although it can 
exceed 90%. By this measure, the efficiency 
of film is very low, since only about 2% of 
incident photons are absorbed by silver hal- 
ide crystals to produce a latent image. As 
the exposure increases, the number of elec- 
trons in the pixel capacitor increases toward 
a maximum capacity somewhere between 
5000 and 100,000 electrons, known as the 
full-well capacity (FWC). The FWC, which 
determines the maximum signal and the 
dynamic range, depends on the physical size 
of the pixel. In CCDs the charges are shifted 
cell by cell to external amplifiers. In contrast 
to this, the electrons accumulated during 
the integration period in a pixel ofa CMOS 
sensor are stored in the silicon potential 
well, and at the end of the collection period 
are converted to a voltage. Each pixel has 
a reset transistor, an amplifier, and row 
and column connections that permit the 
pixel to be read directly. The construction 
details and the performance specifications 
for modern sensors are proprietary and are 
not readily available from the vendors, but 


there is no shortage of individuals eager to 
test the final products. 


16.4 Encoding of Color 


s described above, the pixels in an 
image sensor accumulate electrical 
charge that is proportional to the 
intensity of incident light in a certain fre- 
quency range. This provides information 
about luminance but not about color. In 
order to encode color, we require informa- 
tion about the intensity of light in at least 
three different frequency ranges. Ideally, 
each pixel would contain three independent 
detectors that would mimic the sensitivity 
of the three cones in the human eye, but this 
is not yet possible, and we must consider 
various compromises. One possibility is to 
use a trichroic prism assembly to reflect 
red, blue, and green light from the image 
onto three different image sensors as illus- 
trated in Figure 16.2. Information from the 
three sensors can be brought into register 
and combined in a computer memory to 
simulate an array of three-color pixels. This 
scheme, known as 3CCD, works well and 
has been used in some video cameras and 
camcorders, but it requires three times as 
many elements for the image detector and 
is expensive. 
Another possibility is to use the fact 
that silicon absorbs different amounts of 
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FIGURE 16.2. Philips type trichroic prism assembly (see Wikipedia). 


red, blue, and green light so that different 
frequency distributions can be found at 
different depths in a chip. The silicon acts 
like a set of filters absorbing blue more 
than green and green more than red as 
illustrated in Figure 16.3. This means that 
detectors at different levels in the chip can 
obtain enough color information to permit 
accurate three-color pixels to be simulated. 
The silicon filtering effect is exaggerated 
here since all three detectors receive some 
R, G, and B photons, and only the rela- 
tive amounts are different. This layout of 
detectors is, in fact, the basis of Foveon 
X3 technology that is now being offered in 
cameras made by Sigma Corporation. Each 


pixel is capturing most of the incident light 
by means of three detectors, and the color 
processing uses only information from that 
pixel. Therefore, resolution is, in principle, 
enhanced compared to arrangements where 
some color information must be obtained 
from neighboring pixels. 

Most digital cameras in 2016 encode 
color by using a color filter array (CFA). In 
this scheme each pixel is covered with a filter, 
usually red, blue, or green, and information 
must be combined from a set of neighbor- 
ing pixels to assign R, G, and B values to a 
pixel. In other words, after an exposure each 
pixel has a value (number) that corresponds 
to the intensity of light transmitted by one 


FIGURE 16.3. Illustration of red, green, and blue detectors in each X3 pixel. 
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of the color filters. The array of numbers 
for all the pixels must then be processed 
to obtain a set of three numbers for each 
pixel to specify its full color. This compu- 
tation has been called demosaicing. In the 
simplest algorithm, a pixel is assigned color 
values from neighboring pixels to complete 
the set of three colors. For example, a red 
(R) pixel will be assigned B and G values 
from its neighbors. This gives a fast but 
poor representation of color. The next best 
scheme involves computation of the aver- 
age color values of neighboring pixels. Thus, 
an R pixel gets a B value that is the aver- 
age of two or four neighboring blue pixels. 
Still more sophisticated algorithms, includ- 
ing cubic-spline averaging, are available 
in the integrated circuit inside the camera 
to determine appropriately averaged val- 
ues of three colors for each pixel in the 
sensor. 

Filter arrays turn each pixel into a color 
detector, reminiscent of the operation of 
the retina in the human eye, especially the 
fovea (see Chapter 14). A major difference 
is that CFAs contain periodic patterns of 
filters and, therefore, sample images at reg- 
ular intervals, while the cones in the retina 
have a random distribution. Random dis- 
tributions are also found for grain in film. 
Unfortunately, periodic sampling can pro- 
duce undesirable artifacts such as Moiré 


(a) (b) 


fringes. Ways to avoid this problem are 
discussed below. The use of randomly dis- 
tributed pixels in an image sensor appears 
to be an impractical solution because of 
problems with pixel readout and image pro- 
cessing. The human brain requires years to 
learn how to process the visual signals from 
randomly distributed cones and rods in 
the retina. Of course, this is not a fair com- 
parison because the brain must also deal 
with binocular vision, motion detection, 
coordination of motion, etc. 

By far the most popular filter array for dig- 
ital cameras is the one invented by Bryce E. 
Bayer at Eastman Kodak in 1974 (see Figure 
16.4(a)). The Bayer pattern is 50% green, 
which is reasonable since the human eye has 
its maximum sensitivity and resolving power 
in the green region of the spectrum. In this 
scheme the green elements determine the 
luminance, while the red and blue elements 
provide chrominance information. 

In the second pattern (Figure 16.4(b)) half 
of the green filters are replaced with “emer- 
ald” filters. This pattern, introduced by Sony, 
in principle permits a more accurate repre- 
sentation of color than can be obtained with 
only three filters. As explained in Chapter 
15, any choice of three color primaries can- 
not exactly match all the colors that humans 
perceive. In fact, with some choices of ref- 
erence colors, the matching of a test color 


(c) (d) 


FIGURE 16.4. Color filter arrays: (a) Bayer pattern — RGBG, (b) Sony RGB+emerald pattern, (c) 
Bayer with subtractive primaries, and (d) Kodak RGB+white pattern. 
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in the blue-green spectral region requires 
a negative amount of red light. This simply 
means that a mixture of the blue and green 
reference beams can match the sample in a 
given frequency region only if red light is 
first added to the sample! This longwinded 
explanation is meant to justify the introduc- 
tion of a fourth primary to improve color 
matching over the entire visible region. 

The third pattern (Figure 16.4(c)) is a 
Bayer array with R, G, and B replaced with 
the subtractive primaries, cyan (C), magenta 
(M), and yellow (Y). Kodak used this one in 
an early digital camera. The justification for 
the CMYK array is that these filters trans- 
mit twice as much light as the RGBG array. 
The R, G, and B filters each transmit a maxi- 
mum of 1/3 of the total intensity. In order to 
understand this effect, think of M=B+R, 
Y =R + G, and C = B + G. The improved 
sensitivity may be offset by less effective 
demosaicing algorithms and poorer color 
rendition. Some video cameras use a modi- 
fied CY MY array in which half of the yellow 
pixels are replaced with green in an attempt 
to improve the color quality. 

The fourth pattern (Figure 16.4(d)), is one 
of a group of filter arrays that include clear 
filters to make panchromatic pixels. The 
panchromatic pixels are, of course, more 
sensitive; and they provide the luminance 
information. Kodak pushed this approach 
as a way to improve sensor performance. 
Various patterns, some including 50% pan- 
chromatic pixels, require different levels of 
image processing, and may be appropriate 
for different segments of the camera mar- 
ket. The main advantage of sensors with 
panchromatic pixels appears to be sensitiv- 
ity. The downside is that the sensors behind 
the clear filters tend to saturate first. 

Another filter array pattern from Sony 
(not shown here) offers spatially varying 
pixel exposures (SVPE) to permit HDR 


capture in a single image. Four different 
optical densities ranging from clear to dark 
are applied to the color array so that one 
fourth of the green pixels has zero added 
density, one fourth has the next higher den- 
sity, and so on. This also applies to the red 
and blue pixels. Therefore, some pixels of 
each color receive the proper exposure for 
each part of the range of bright to dark parts 
of the image. Algorithms then permit the 
light and dark pixels to be used to create a 
high-resolution image with preserved high- 
lights and shadows. 

In another innovation, Fujifilm’s X-Trans 
color filter array has an increased repeat 
distance and moves a step closer to a ran- 
dom color filter array. This was an attempt 
to decrease color moiré effects relative to 
Bayer filters. The change justified making 
the X-series cameras without anti-aliasing 
filters. The real problem is that the best 
lenses may resolve more detail than the 
sensors can properly sample. As pixel count 
increases, aliasing will eventually cease to 
bea problem. (See Section 17.8.) 

Sensor development will continue to be 
an active area for years to come. In addition 
to the steady increase in pixel count, there 
are reports almost monthly of new schemes 
to improve sensitivity and dynamic range. 


16.5 How Film/Sensor Speed 
(ISO) Is Defined 


ilm “speed” refers to the film’s sen- 
Es to light and how quickly a 

good exposure can be obtained. 
Speed and ISO are used interchangeably, 
but ISO refers to a set of standards pub- 
lished by the International Organization 
for Standardization. ISO and the older 
ASA standards are essentially the same 
for film. Unfortunately, the whole area of 
film and sensor standards is complicated 
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and tedious. However, two points merit 
our interest. First, on what measurements 
are standards based and, second, how do 
film and sensors differ in their response to 
photons? The quantitative characteriza- 
tion of film dates from the 19th century, 
when Ferdinand Hurter and Vero Charles 
Driffield started studying plate density as a 
function of exposure. A characteristic curve 
(H and D curve) for monochromatic film 
is shown in Figure 16.5. The development 
time determines the steepness of this curve, 
so the standard is based on film developed 
to the point that the optical density (OD) 
changes by 0.8 units from the 0.1 line when 
the logarithm of the exposure increases by 
1.3 units. (The illuminance is measured in 
units of lumens (/x), defined in Appendix 
I.) This shape can be obtained regardless of 
film speed, but the numbers on the abscissa 
will, of course, depend on the film. Any film 
will have some minimum density that arises 
from the cellulose base and emulsion fog. 
Therefore, to avoid the density not associ- 
ated with exposure, the characterization of 


the film begins at 0.1 OD units above the 
intrinsic density; and film speed is based on 
the exposure at the point m, denoted by H 
that is required to reach that level. 

For monochromatic film the ISO speed 
is defined as S = 0.8/H,, and a good expo- 
sure for an average outdoor scene would 
correspond to roughly H, = 9.4/H, . The 
procedure is more complicated for color- 
negative film, since there are three layers, 
and color-reversal film requires a different 
reference point. In all cases, however, the 
speed is based on the exposure required to 
have a defined effect on the density of the 
developed emulsion. After the film speed 
has been established according to the ISO 
standards, photographers are able to set their 
cameras to obtain consistent exposures. 

The idea behind the concept of speed 
ratings is that total exposure determines 
the density, and the exposure is always the 
product of illuminance and time. This is 
the assumption of reciprocity, namely, that 
there is an inverse relationship between the 
intensity and duration of light required to 
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FIGURE 16.5. The characteristic (H and D) curve for monochromatic film. Optical Density versus 


Exposure, H in lux seconds. 
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have a defined effect on film, and photons 
all count the same regardless of how fast 
they arrive. In real life, the reciprocity law 
fails for exposures less than about 0.001 
sec and greater than a few seconds. At very 
short exposures, many photons arrive at 
almost same time, and they are not used 
efficiently. With very long exposures there 
is low-intensity reciprocity failure. In this 
limit, photons are apparently lost or some- 
how forgotten. 

The H and D curve reveals other impor- 
tant characteristics of film. In the toe region, 
starting from zero exposure, we see that 
many photons are required before there is 
a measurable change in the density. Next is 
the linear region, the most useful part for 
creating images. The slope in this region, 
defined as y = A(OD)/A(log H), determines 
the contrast of the image. For monochro- 
matic negatives with normal development, 
gamma is about 0.7. At still higher expo- 
sures, we encounter the shoulder and finally 
the solarization region, where additional 
exposure actually decreases the density. 

Now consider the characteristics of 
sensors, either CCD or CMOS. A photo 
detector (a photodiode) in each pixel cap- 
tures photons efficiently and activates up 
to one electron for each photon. The exact 
number of photons per electron is defined 
as the quantum efficiency of the detector. 
The activated, or free, electrons accumulate 
during the integration period in a potential 
well and constitute an electrical charge that 
can be converted into a voltage. Think of the 
electrons as filling a well so that the number 
of electrons at any time is proportional to 
the exposure. To a very good approximation 
this is a linear process, and for all practical 
purposes the reciprocity law is satisfied. Of 
course, at extremely short and long times 
there will be variations, but that need not 
concern us. The characteristic curve of the 


number of electrons versus the exposure 
for sensors is shown in Figure 16.6. In this 
plot the plateau on the right side represents 
the full-well capacity (FWC), or saturation 
number, of electrons. 

The speed of a sensor is defined by the 
Standard Output Sensitivity (SOS), roughly 
equivalent to the ISO speed, or by the 
Recommended Exposure Index (REI). SOS 
and REI are standards published by the 
Camera and Imaging Products Association 
(CIPA).* The SOS speed S is based on the 
exposure H „that is required to obtain 0.461 
of the maximum digital output under pre- 
scribed illuminance conditions, and the 
speed is defined as S = 10/H,. (The factor 
0.461 is deemed to give a medium output 
which corresponds to the relative output 
of an 18% reflective standard on an sRGB 
gamma curve.) Now, suppose that the signal 
from the pixel is amplified by a factor of two 
before it is digitized so that a full scale read- 
ing (maximum brightness) can be obtained 
with only half the FWC. As a result, the 
measured value of H, will be a factor of 
two smaller and the SOS (or ISO) number 
will double. The REI is similar, except that 
H, is replaced by the manufacturer's rec- 
ommended average exposure in the focal 
plane. Still other definitions have been pro- 
posed, for example, S,, = 78/H,,, where H,, 
is the exposure that causes clipping of the 
signal. The minimum value of S „is deter- 
mined by the sensor, but higher values can 
be obtained with amplification before the 
analog to digital (A/D) converter. There 
are also the noise-based sensor speeds 
with reference exposures that produce out- 
put signals 10 or 40 times the noise in the 
sensor. 

The point of these standards is to estab- 
lish a number that will yield images similar 
to those obtained with film at the same ISO 
setting. So what do camera manufacturers 
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FIGURE 16.6. Characteristic curve for a sensor: 


exposure H in lux seconds. 
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electrons accumulated in a pixel well versus 


actually report for the SOS/ISO number? It 
is not always clear. It would be reasonable to 
use the 10:1 noise result as the upper limit 
and 40:1 noise result or the H „based speed 
as the standard value. It is clear, however, 
that high SOS values are associated with 
lower quality images, and the acceptable 
noise is a subjective judgment. 


16.6 How the Dynamic 
Range Is Determined 


he dynamic range is the ratio of the 

maximum to the minimum signal 

that can be measured. In photog- 
raphy signal refers to the light sensitivity 
range of films or sensors. It could also mean 
the opacity range of negatives or the reflec- 
tance range of prints. 

The minimum signal can be thought of 
as read noise, where read refers to obser- 
vation or measurement. In science, noise is 
defined as fluctuations that accompany sig- 
nals of interest. Noise is random and usually 


undesirable. Most people will think of static 
ina radio signal, but the “snow” ina TV pic- 
ture is also noise. The signal-to-noise ratio 
(SNR) is a useful parameter for defining 
the quality of a signal or an image. It might 
be thought that the maximum SNR ratio 
would be the same as the dynamic range. 
For digital cameras this not the case, though 
both quantities depend on the FWC. I will 
return to this point later. 

The weakest image that can be discerned 
on photographic film is one that just exceeds 
the fluctuations resulting from the grain. 
Grain is nothing more than the silver parti- 
cles or clouds of dye molecules that make up 
the image, and graininess is our subjective 
assessment of noise in the image. On the 
other hand, granularity is a numerical value 
that is determined with a microdensitom- 
eter. Root-mean-square (rms) fluctuations 
are computed for the fluctuations in optical 
density measured with a 48 um aperture, 
and the result is usually multiplied by 1000 
and reported as the granularity. 


176 cy CHAPTER 16 


Now let’s look at sensors. The maximum 
signal is determined by the FWC. Therefore, 
the dynamic range is defined as the FWC 
divided by the read noise. The read noise 
contains all of the on-chip noise that is inde- 
pendent of the photon signal. This noise 
results from the readout amplifier as well as 
thermal electron generation (dark noise). 
Dark charge accumulation can be reduced 
by perhaps a factor of 10 by cooling a CCD 
sensor by 10°C. The average dark count can, 
of course, be subtracted out. CMOS sensors 
have more on-chip components and more 
sources of noise in addition to dark noise. 
There are reset transistor noise (error), 
amplifier noise, photodiode leakage, etc. 
However, the dynamic range can be com- 
puted the same way. 

As the photon signal increases, it soon 
overwhelms the chip noise, but it brings 
with it significant statistical noise. There 
is an average arrival rate for photons, but 
the arrival time of a photon is independ- 
ent of the time since the last arrival. This is 
similar to the arrival of rain drops on a leaf 
or patrons at a box office. The distribution 
of arrival times is described by the Poisson 
distribution, and the important conclusion 
for this application is the following. If the 
mean number of events (arrivals) is equal 
to n, the number of arrivals fluctuates about 
the mean value with a standard deviation 
equal to o = Vn. If an exposure is made of a 
uniform gray card with a digital camera, the 
pixel potential wells will contain some aver- 
age number of electrons, say n. Therefore, 
the signal is proportional to n while the 
noise is proportional to Va, and the SNR 
is just (n/Vn) = Vn. As a concrete example, 
suppose that the FWC is 20,000 electrons 
and the read noise is 10 electrons. The shot 
noise associated with the FWC is v20,000 
electrons and the SNR is 141 while the 
dynamic range is 20,000/10 = 2000. In fact, 


the true SNR would be slightly less since I 
have only taken into account the photon shot 
noise. Also, the dynamic range, while useful 
for comparison, is not a practical number, 
since a signal having the magnitude of the 
read noise could not be distinguished. A 
recognizable signal would require a SNR of 
about 3. 

It is quipped that film is digital and sen- 
sors are analog. There is some truth in this 
because a film grain is either “on” (black sil- 
ver grain) or “off” (transparent), while the 
pixels in sensors store various amounts of 
charge that can be converted to a voltage 
and later digitized. This brings us to the 
question of how many bits are required for 
the digitization. As an example, consider 
the Canon S70 compact camera. Roger N. 
Clark analyzed this camera and reported 
that for ISO 50: FWC = 8200 electrons and 
(read noise) = 4.1 electrons.* Therefore, the 
dynamic range is 8200/4.1 = 2000 and the 
maximum SNR is ¥8200 = 91. The digiti- 
zation must adequately cover the dynamic 
range of the sensor, and that should deter- 
mine the number of bits required. The 
number of steps provided by digitization 
increases in the procession 2” where n is the 
number of bits. Starting with n = 8 and stick- 
ing to even numbers, we find the sequence: 
28 = 256, 2!° = 1024, 2 = 4096, 2'* = 16,384. 
So it appears that 12-bit digitization is 
completely adequate. The extra bit-depth is 
actually very useful when images are pro- 
cessed and mathematically manipulated to 
selectively enhance brightness and contrast. 
With marginal digitization, image process- 
ing can result in gaps in the histogram of 
signal versus luminosity. 

The Canon S70 shows high image qual- 
ity at ISO 50, but at ISO 200 the number 
of electrons for the maximum signal drops 
to 2150. This means that a full scale read- 
ing requires ISO amplifier gain of 2.0 for 
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the signal and the noise with loss of image 
quality. Also, the SNR drops to 45. However, 
the calculation of the dynamic range is not 
straightforward, because the effective read 
noise is not independent of the ISO setting. 
Note that the ISO amplifier gain is not the 
same as the camera gain. The camera gain 
is highest at low ISO settings and refers 
to the number of electrons per analog-to- 
digital unit (ADU) or data number (DN). 
For example, with 12-bit digitization the 
number of electrons required for full scale 
reading would be divided into 4096 steps to 
obtain the DN. 

The discussion here seems to indicate 
that an increase in the ISO (gain) set- 
ting automatically causes a proportional 
decrease in the dynamic range of an image. 
Similarly, decreasing the ISO setting by 
one stop is expected to double the dynamic 
range. This is not the case for all sensors. 
The difference occurs because read noise 
has two parts, and only one of them is 
affected by the ISO setting. Photon cap- 
ture noise and other sensor noise, that are 
acquired before the ISO amplification, are 
known as upstream noise. Sensor noise 
added after amplification is designated 
downstream noise. Upstream noise is gen- 
erally low, and at low ISO settings it usually 
makes a minor contribution. The amount 
of downstream noise varies considerably 
among manufacturers. 

As the ISO setting is increased, the 
amplified upstream noise will at some point 
dominate over downstream noise and will 
determine the dynamic range. Therefore, 
sensors with different levels of downstream 
noise may register the same dynamic range 
at high ISO. The total read noise arising from 
both ISO amplified upstream noise and 
downstream noise will certainly increase as 
the ISO amplification is increased. However, 
for sensors with low downstream noise, the 
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increase in read noise will be relatively lin- 
ear, and the decrease in dynamic range will 
be linear. Also, the dynamic range at base 
(unamplified) ISO will be higher. These 
sensors are defined as ISO invariant. 

So what are the advantages of an ISO 
invariant sensor? Suppose that a backlighted 
subject is photographed with the exposure 
1/50 s at f/8 and with an ISO setting of 100. 
If the subject is extremely underexposed, 
the image can be brightened by increas- 
ing the ISO amplification by several stops. 
With five stops increase, the image will be 
32 times brighter. The other alternative 
is to leave the ISO at 100 and to brighten 
the image by five stops in processing with 
Photoshop, Lightroom, or some other soft- 
ware. If the sensor is ISO invariant, either 
procedure will work. Actually, brighten- 
ing in software may be preferable because 
brightening can be selective so that the 
already bright background can be retained. 
If brightening is accomplished by using ISO 
3200, the background will be blown out and 
processing options will be more limited. 

Suppose instead that the sensor is not 
ISO invariant, and that the dynamic range 
is relatively low at ISO 100. Brightening the 
image by 5 stops in processing may lead 
to unacceptable noise in the final image. 
In this case, it would be preferable to do 
the brightening in camera with increased 
ISO amplification. ISO invariance with the 
accompanying high dynamic range at base 
ISO is clearly desirable. This is especially 
true for landscape photographers, who like 
to work at low ISO settings and who need 
high dynamic ranges to recover shadow 
detail. However, the advantage may be small 
or nonexistent for sports and wildlife pho- 
tographers, who usually work at high ISO 
setting, and there may be trade-offs in sen- 
sor design that work against ISO-invariant 
designs. 
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FIGURE 16.7. Dynamic range (DR) versus ISO amplification: ISO invariant sensor in Sony A7R 
(red) and non-ISO invariant sensor in Canon 6D (black). 


Typical plots of dynamic range versus 
ISO are shown in Figure 16.7 for a sen- 
sor that is close to ISO invariant (red) and 
a sensor that is clearly not ISO invariant 
(black). These were measured for two FF 
cameras on the market in 2016.° It is inter- 
esting to note that the non-ISO invariant 
sensor actually has higher dynamic range at 
very high ISO than the ISO invariant sen- 
sor. It would clearly be preferable to have 
high dynamic range and ISO invariance 
for all ISO values, but that may be hard to 
achieve. Extensive testing also shows that, 
for some almost ISO invariant sensors, the 
initial ISO setting does affect the noise level 
of the final processed image. For these sen- 
sors, there is a “sweet spot” in the ISO range 
for best results. 

I end this section with some estimates 
of the dynamic ranges associated with vari- 
ous imaging systems. These numbers have 
large errors because estimates of noise and 
visual quality are subjective and technology 
continues to improve. As before, one stop 
represents a factor of two and n stops means 
a factor of 2”: 


Sensors (DSLR): 11-15 stops 
(RAW readout), 8—9 
stops (JPG readout) 
Color print film: 7 stops 
Color slide film: 5 stops 
Human eye: 5-7 stops (fixed 


retina, without iris 
change) 


Of course, much higher dynamic ranges 
can be captured with digital techniques by 
combining several images taken with differ- 
ent exposures. This is the basis of the high 
dynamic range (HDR) technique that was 
demonstrated in Chapter 13. The human 
eye also uses a type of HDR method when it 
supplements the capabilities of the retina by 
varying the diameter of the iris in response 
to the brightness of selected parts of a scene. 


16.7 Exposure Settings for 
High Image Quality 


e have seen that exposure is 
determined by aperture and 
shutter speed. The expo- 


sure meter on the camera and the image 
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brightness also depend on the ISO setting. 
All of these settings can be accomplished 
automatically in some cameras by selecting 
the AUTO setting. That will insure the cap- 
ture of a JPG image with F-number, shutter 
speed, and ISO number preselected to give 
a reasonably bright image. The user may 
also have the choice of default adjustments 
of saturation, sharpness, and white balance. 
Everything is, of course, taken care of on 
your cell phone with no user input at all, 
and the results obtained will be satisfactory 
for most users. 

However, the reason for buying a sophis- 
ticated camera is to have complete control 
of all the parameters that affect image qual- 
ity. We have seen that the aperture affects 
the depth of field and that the shutter speed 
determines how motion will be captured. 
In addition, the ISO setting determines the 
dynamic range as well as the signal-to-noise 
ratio. Furthermore, the choice of settings 
is not unique and requires knowledge and 
judgment. Many cameras, and all inter- 
changeable lens cameras, offer a choice of 
automated, semi-automated, and manual 
settings. 


In principle, serious photographers 
should choose each setting to optimize the 
features they wish to capture. The selec- 
tion of F-number and shutter speed will, 
of course, limit the range of ISO settings 
for acceptable image brightness. There are 
more choices to be made when the dynamic 
range of the sensor exceeds the dynamic 
range of the scene that is photographed. 
Figure 16.8 shows a set of RAW images in 
a three-shot bracket with a set of different 
shutter speeds at fixed aperture and ISO 
settings. The lower part of each of the three 
frames shows the exposure histogram, i.e. 
number of pixels vs. brightness level. The 
right end represents full-well capacity. At 
this and any greater exposure, the bright- 
est part of the image will be clipped, and all 
information will be lost. The left end corre- 
sponds to zero exposure, and pixels at this 
end will provide no detail in the dark areas. 
Between these two limits the histogram 
indicates the number of pixels in the image 
at each brightness level. 

Starting with the (a) panel on the left 
side, we find a histogram that fits fairly 
well into the dynamic range of the sensor. 


(a) (b) 


(c) 


FIGURE 16.8. Images obtained with a camera on a tripod in Arches National Park: (a) 1/250 sec, 


(b) 1/100 sec, and (c) 1/30 sec. 
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Perhaps some detail has been lost in the 
shadow areas, but most of the image can be 
brightened to taste in the processing step. 
The center panel (b) shows a better expo- 
sure with little information loss at either 
end of the brightness range. Furthermore, 
more photons were captured in each area 
of the scene than was the case in image (a). 
This indicates that less brightness adjust- 
ment will be required in processing; and 
that the signal-to-noise ratios will be higher, 
especially in the darker areas of the scene. 
The panel on the right side, (c), illustrates 
an overexposed image; and the associated 
histogram shows clipping on the right side. 
Detail has been lost in the in the sky, and 
nothing can be done to retrieve the infor- 
mation in image processing. However, all 
the images can be combined in HDR pro- 
cessing software to provide the best possible 
result, with complete shadow detail from 
the brightest image and complete sky detail 
from the darkest image. 

A comparison of images (a) and (b) sup- 
ports the argument for exposing to the right 
(ETTR). It is always good to capture more 
photons in each pixel so that noise can be 
minimized. It is also true that RAW images 


captured with ETTR sometimes appear 
flat and even a bit overexposed right out of 
the camera, but that is no problem since the 
aim is to optimize the quality of the final 
processed image. Disadvantages of ETTR 
include the danger of clipping high lights 
and the inconvenience of examining histo- 
grams. Exposure bracketing, when possible, 
can avoid these problems. The most impor- 
tant lesson is that clipped images should be 
avoided except when an exposure bracketed 
set is being captured for HDR processing. 
There is still the question of semi- 
automated exposure settings. A typical 
mode dial from a digital camera is shown 
in Figure 16.9. Here B or bulb indicates that 
the shutter will be held open as long as the 
shutter is activated, and the manual setting 
(M) is self-explanatory. The other three set- 
tings on the right-hand side remain to be 
explained. In the following I assume that 
ISO has been set elsewhere in the camera 
either at a desired value or in the automatic 
mode to select a value consistent with the 
chosen aperture and shutter speed. Av indi- 
cates aperture priority. This means that the 
appropriate F-number is user-selected and 
the shutter speed is selected automatically 


FIGURE 16.9. A typical mode dial. 
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for the chosen ISO value. Tv indicates shut- 
ter priority. Here the shutter speed is user 
selected, and the F-number is set automati- 
cally to a value consistent with the ISO. 
Finally we reach P which stands for pro- 
gram mode. P is the most automatic of the 
settings short of completely automatic (A). 
In this mode a consistent pair of F-number 
and shutter speed settings is automatically 
set for the chosen ISO value. Details of the 
implementation may depend on the camera 
maker, but usually image format (RAW vs. 
JPG), exposure compensation (explained 
below), exposure bracketing, and flash con- 
trol are user selected. The P mode selects 
an acceptably short shutter speed when 
possible, and there is usually a way for the 
user to cycle through the set of consistent 
F-numbers and shutter speeds and select 
another pair if desired. 

The automated modes are implemented 
in different ways by the various camera mak- 
ers. Exposure compensation and exposure 
bracketing are particularly useful features. 
The camera exposure meter shows the sta- 
tus of the predicted image brightness from 
—3 stops underexposed to +3 stops overex- 
posure as shown in Figure 16.10. Part (a) 
shows the normal exposure set by any of 
the exposure modes. In part (b) an exposure 
compensation of stop has been dialed in, 


Q 


and in part (c) a three-stop bracket has been 
dialed in. The bracket will be accomplished 
by automatically adjusting the shutter speed 
as illustrated in Figure 16.9 if aperture 
priority has been selected. 

The camera exposure meter shows the 
effect of the ISO setting on image bright- 
ness even though ISO amplification of the 
signal strength does not affect exposure. 
The effect of ISO is similar to that of emul- 
sion speed in film photography. However, 
the range of ISO settings and the AUTO 
ISO feature offer great flexibility and con- 
venience to photographers. For example, a 
photographer can select a shutter speed to 
capture motion, an aperture to obtain the 
desired depth-of-field in manual mode, and 
then count on AUTO ISO to ensure the 
appropriate brightness. Additional feature 
sets, offered on some cameras, may permit 
the range of ISO values to be limited in the 
AUTO mode and may even ensure that the 
effect of the exposure compensation setting 
is not cancelled by AUTO ISO. 

This discussion would not be com- 
plete without a consideration of how the 
image exposure is sampled by the exposure 
meter. Cameras offer a choice of exposure 
sampling modes, and the choices usually 
include center weighted, spot, and matrix/ 
evaluative metering. Each mode selects 
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FIGURE 16.10. Exposure meter showing (a) normal exposure, (b) +1 stop, and (c) —, o, +1 stop 


bracket. 
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image zones for weighting (influencing) 
the meter reading. Matrix metering is the 
most sophisticated since the point of focus 
influences the metering. Spot metering 
is obviously the most selective, but in a 
given case may not be selective enough or 
may not be aimed at precisely the desired 
areas. There will always be cases where the 
meter reading is misleading. For example, 
snow scenes lead to meter readings where 
the snow is assigned a medium gray level 
and other objects in the scene are greatly 
underexposed. This can be corrected by an 
exposure compensation of one to two stops 
to brighten the snow. A similar effect occurs 
when trying to photograph birds in flight 
with bright sky as the background. One way 
to handle this situation is to set the proper 
exposure in manual (M) mode for the bird 
on the ground, and then to ignore the meter 
readings when the bird is in flight. 


16.8 Saving and Storing the Image 


Images right out of the camera—JPG and 
RAW: Essentially all digital cameras per- 
mit images to be displayed and transferred 
in the JPEG (JPG) format. This format was 
developed by the Joint Photographic Experts 
Group and approved in 1994. It is extremely 
convenient for users because image process- 
ing is done in the camera and the images 
are delivered ready for viewing or printing. 
Furthermore, JPEG is a compressed file for- 
mat that reduces the memory requirements 
for storing images. The images are com- 
pressed by a user-selected factor and then 
stored. The stored images can then be recon- 
structed for viewing. File size reduction by 
a factor of 10 yields an image that is hard to 
distinguish from the original, but reduction 
by a factor of 100 produces obvious artifacts. 

Unfortunately, JPEG encoding has draw- 
backs, and it is not the best choice for high 


quality photography. Here are the major 
problems: 


1. The color balance, contrast, sharpness, 
saturation, and JPEG quality must be 
set in the camera before the photo- 
graph is taken. 


2. The image is encoded with only 24 bits 
total or 8 bits per color. 


3. Compression is “lossy” and always 
reduces image quality to some extent. 


In the JPEG image all the settings are 
locked in. Of course, post-processing with a 
program such as Photoshop can make some 
corrections; but it is very difficult, if not 
impossible, to obtain true color balance if 
the original white balance setting was incor- 
rect. Furthermore, the 8-bit image does not 
respond well to multiple post-processing 
steps to enhance image brightness and con- 
trast. For example, 8-bit digitization may 
have been good enough to describe the dis- 
tributions of electrons in the pixel wells in 
the sensor, but a processing step can stretch 
the brightness range so that gaps appear for 
signals with only 8 bits of information. 

The way around these problems is to 
select a RAW format rather than JPEG. 
RAW is available on some point-and-shoot 
cameras and all interchangeable lens cam- 
eras. In principle, RAW captures all of the 
information from the sensor pixels at the 
maximum resolution of the A/D converter, 
usually 12 or 14 bits. At this level, each 
pixel is still associated with a single color, 
usually R, G, or B. Therefore, a conver- 
sion program is necessary for demosaicing, 
sometimes called “developing, the image 
to obtain three-color pixels. Such programs 
are available from camera vendors as well 
as in general purpose photo processing 
packages. Camera RAW (ACR) in Adobe 
Photoshop is an example of a program for 
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“developing” RAW files. The real advantage 
here is that the conversion programs permit 
the user to set the white balance, sharpen- 
ing, contrast, and even, to some extent, the 
exposure after the fact, taking advantage of 
the full dynamic range encoded in 12 or 14 
bits. The converted file can then be saved 
in a standard format such as lossless TIF, 
and the original RAW file is unchanged. 
The RAW file acts like a “digital negative” 
for archival purposes. 

But are RAW files really unprocessed 
data? First of all, we know that they are 
compressed, but we hope the compression 
is lossless or almost lossless. There is also 
the question of how camera manufacturers 
deal with bad pixels (dead or “hot” pixels). 
Those pixels, which inappropriately add 
to the recorded image, produce fixed pat- 
tern noise that can be removed by means of 
dark-frame subtraction. In this procedure, 
photographs are taken with the shutter 
open and then with the shutter closed. The 
dark-frame image is subtracted from the 
shutter open image to remove the contri- 
bution from the unchanged pattern noise. 
It is also possible that the camera has been 
programmed by the manufacturer to do 
some automatic image manipulation to “fix” 
pixel errors. This is usually not of concern, 
but astrophotographers report that some 
older Nikon cameras appear to remove the 
images of stars as part of automatic pixel 
clean-up even when preparing RAW files.* 


16.9 Image Processing 
and Manipulation 


AW or JPG images from cameras 

P must be processed before they can 
be displayed or printed. Typically an 

image from a camera’s memory card is cop- 


ied to an appropriately named file folder on 
a hard disk or solid state drive in a computer. 


The image file then is read by or imported 
into image processing software. All process- 
ing software can handle JPG files; but RAW 
files come in a variety of proprietary formats, 
depending on the camera manufacturer and 
camera model, and may not be supported. 
There is an alternative RAW format known 
as Adobe digital negative (DNG), which 
was develop by Adobe Systems to provide 
a universal format. However, only a few 
camera manufacturers have adopted DNG. 
DNG offers smaller file size and some insur- 
ance against manufacturer support being 
dropped for formats specific to out-of-date 
cameras; and any RAW format can be con- 
verted to the DNG format. Thus the DNG 
option remains open to any photographer. 
The main complaint against DNG is that it is 
not universally supported by either cameras 
or imaging processing software. This may 
well change with the acceptance of DNG by 
some cell phone manufacturers. 

In any event RAW files require addi- 
tional processing. The files can be converted 
to either JPG files or Tagged Image Format 
Files (TIFF). TIFF files, which can be either 
8 bit or 16 bit per channel, support multiple 
file layers and offer a superior alternative for 
RAW conversion. The RAW conversion can 
be accomplished with programs from cam- 
era manufacturers, such as Canon, Nikon, 
and Olympus, or by third party packages 
such as Adobe Camera RAW, DxO Optics 
Pro, and Capture One Pro. It is also possi- 
ble to avoid file conversion and to perform 
image corrections and enhancement on the 
RAW image. In Adobe Lightroom the origi- 
nal RAW file is not changed, but rather an 
Extensible Metadata Platform (xmp) sidecar 
file is created containing a list of instruc- 
tions for development and modification to 
be used when the RAW file is exported or 
printed. When the DNG format is used, the 
xmp instructions are incorporated and a 
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separate file is not required. In any case the 
original RAW data are not disturbed. 

In Lightroom one can maintain rela- 
tively small RAW files, can edit the changes 
at any time, and can create virtual copies 
without additional image files. This kind of 
processing is known as parametric editing. 
The processing, however, is limited com- 
pared with what can be done in Photoshop 
and perhaps other image editing software. 
Lightroom does not support either layers 
or extensive cloning. Lightroom does offer 
very useful database management. Dates 
and camera and lens types can be searched, 
keywords can be added to images, and vir- 
tual collections of images can be created. 
Furthermore, lens profiles can be applied to 
correct for distortions and vignetting, and 
camera profiles can be created to correct for 
the rendition of colors by a specific sensor. 

The subject of image processing is 
extensive and is treated in many books. 
In addition, the capabilities change each 
year with the introduction of new software 
packages. The new user is advised to take 
advantage of instructional videos to learn 
what is available as well as how to use the 
features. 1 have benefited from Lightroom 
videos from the Luminous-Landscape.com 
and Photoshop tutorials from KelbyOne. 
There are also extensive tutorial videos from 
Lynda.com and other instructional sites. 

My typical workflow with Lightroom 
includes the following: 


1. Import image and rename file. 


2. Create new keyword if necessary and 
keyword the image. 


3. Select camera profile from provided 
list or use a custom profile for the 
camera. 


4. Apply lens profile corrections. 


5. Adjust white balance if necessary. 


6. Adjust exposure, and correct the high- 
lights, shadows, whites, and blacks. 


7. Adjust clarity, vibrance, and saturation. 
8. Adjust contrast or the tone curve. 


9. Sharpen the image with mask to pro- 
tect the flat (low contrast) areas. 


10. Apply luminance noise reduction. 


11. Apply local corrections as needed 
with gradient tool and brush. 


These corrections are automatically writ- 
ten into the associated xmp file and saved in 
the folder with the RAW image. File infor- 
mation, settings, and image previews are 
saved in a Lightroom catalog. This catalog 
can be backed up in more than one location. 

This workflow list is not exhaustive, but 
it is probably daunting to beginners. I have 
included it as an indication of the topics that 
need be learned in order to take advantage 
of RAW conversion, database, and image 
processing software. 


16.10 Artifacts Resulting from 
Inadequate Periodic Sampling 


s a concrete example of periodic 

sampling, consider the Canon 40D 

camera. The sensor contains 10.1 
megapixels arranged as a rectangular array 
with a width of 3888 horizontal pixels in 
22.2 mm and a height of 2592 vertical pix- 
els in 14.8 mm. It is common to refer to this 
sensor as having a resolution of 10.1 MP. 
This is incorrect! Resolution is the ability 
to distinguish nearby points or lines, and 
it depends on one-dimensional analysis. 
For the Canon 40D the vertical resolution 
should be related to the pixel density in the 
vertical direction, and in general the reso- 
lution is proportional to the square root of 
the sensor size when sensors with the same 
aspect ratio are compared. 
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The actual resolving power of the sen- 
sor is somewhat less than might be thought 
by simply reading the pixel density. First, 
there is the problem that the sensor sam- 
ples the image in a periodic manner, and 
this can lead to artifacts if the sampling rate 
(samples/mm) is less than twice the highest 
frequency (lines/mm) in the image. This is 
an example of the Nyquist theorem, which 
will be considered in more detail in the next 
chapter. Here it suffices to say that when the 
frequency of sampling is too low, there is 
an apparent down-shifting of frequencies 
in the image, an effect known as aliasing. 
A consequence is the appearance of moiré 
fringes or false periodic patterns. It might 
be thought that this problem would only 
appear when periodic patterns of lines, such 
as weave in fabrics or bricks in walls, are 
photographed. However, all images can be 
described as sums of periodic patterns with 
different frequencies (lines/mm), ampli- 
tudes, and orientations. For example, sharp 
edges can only be described accurately if 
high-frequency components are present. 

It can easily be verified that moiré fringes 
usually do not appear in images taken 
with most digital cameras. More often, 
the fringes appear on a computer LCD 
when small images are displayed, because 
that does lead to under-sampling without 
correction. So how do the camera manufac- 
turers avoid the fringes? The universal cure 
is to filter the image that reaches the sen- 
sor to remove the high frequencies. That’s 
right, the images are artificially blurred 
with an “anti-aliasing” filter! This can be 
accomplished in just the right amount with 
thin layers of a birefringent optical material 
(usually lithium niobate) arranged to split 
an incident beam into four closely spaced 
beams. It seems a shame to effectively 
reduce the resolution provided by a high 
quality lens to avoid sampling errors, but 


there is no alternative. When manufacturers 
fail to provide adequate filtering, fringing 
effects can be detected in resolution tests. 
The sensor tests reported on dpreview.com 
sometimes note the appearance of moiré 
fringes. As pixel densities have increased 
this problem has become less important, 
and some manufacturers now omit anti- 
aliasing filters in an attempt to maximize 
resolution. 


16.11 Gamma Correction 


olor management is an important 

part of digital photography because 

we all want our images to look the 
same regardless of whether they are dis- 
played on a computer, projected onto a 
screen, or printed. Ina RAW image the pixel 
values are proportional to the exposure, and 
after RAW conversion the numbers are still 
approximately proportional to the exposure. 
The human eye has a logarithmic response 
to light and computer screens usually follow 
a power-law dependence. In other words, 
on computer displays the relative intensity 
Lis related to the relative input voltage Vas 
follows: 

I=(VY (16.1) 
where both J and V values are measured 
on a scale between 0 and 1 and the expo- 
nent, y, is usually about 2.2.” This response 
is shown by the lower curve in Figure 16.11. 
Therefore, a mid-range voltage of 0.5 is rep- 
resented by the relative intensity (0.5)?? = 
0.218. This transformation leads to dark 
images, and a correction is clearly called 
for. One way to take care of this response 
is to apply a “gamma stretch” to the input 
voltage to obtain a new input voltage equal 
to VW. The voltage with gamma stretch is 
illustrated by the dotted curve. The exact 
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FIGURE 16.11. Intensity (I) versus the input voltage (V) for a typical computer screen with y = 
2.2. The “stretched” input voltage is shown in the dotted curve. 


correction depends on the color space 
selected, such as sRGB or Adobe RGB. Also, 
some computers and video cards have built- 
in correction. 

Serious photographers often use com- 
mercial calibration systems for their 
computer displays, projectors, and print- 
ers. For example, there are units such as 
the X-Rite Colormunki and the Datacolor 
Spyder5PRO spectrophotometers and 
colorimeters to monitor the brightness 
of the screens or prints for various RGB 
combinations. 


16.12 Capturing the Light Field 


onventional cameras, either film or 
digital, record a two-dimensional 
map of the luminance of a three- 
dimensional scene. The resulting photograph 
displays the intensity of light falling on each 
picture element (pixel) during the exposure. 
This is a very small part of the total infor- 
mation contained in the light field, which 
is defined as all of the light rays passing 


through every point in space in every direc- 
tion.’ No single camera can capture the 
entire light field, but a valuable subset (the 
4D light field) can be obtained if the direc- 
tions of all the rays reaching each pixel can 
be determined in addition to the total inten- 
sity. A hand-held camera with this capability 
was developed at Stanford University and 
reported in 2005 by Ren Ng, et al.? 

The heart of the Stanford plenoptic 
camera is an array of micro lenses one focal 
length behind the main lens. In turn each 
micro lens projects an image of the camera 
lens on an array of photo detectors. Each 
micro lens determines the position x,y of 
the picture element in the final image and 
the photo detectors behind it record the 
angles of the rays as illustrated in Figure 
16.12. Of course, this would mean noth- 
ing without the algorithms and computing 
power that permit images with selective 
focus, variable perspective points, and even 
3D presentation to be extracted. Lytro, Inc. 
has, in fact, produced a consumer plenoptic 
camera.” 
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FIGURE 16.12. Schematic (not to scale) of a plenoptic camera. The detectors capture rays from 
different parts of the camera lens. Note: The object distance is much greater than F and F is 
much greater than f. f is the focal length of the micro lenses and may be of the order of 0.1 mm. 


Photography is always a set of compro- 
mises, so what is the down side with current 
implementations of the plenoptic camera? 
First of all, we recognize that the number of 
micro lenses is much less than the number 
of photo-detectors. That immediately lim- 
its the resolution. Todor Georiev and others 
have suggested modifications in camera 
design to enhance the spatial resolution," 
and an implementation of light field pho- 
tography that does not require additional 
lenses has been demonstrated.’”* 


16.13 Does the Sensor Really 
Determine the Resolution of My 
Digital Camera? 


f everything else is perfect, yes, the sen- 

sor determines the resolution. Actually, 

resolution is such a complex issue that a 
separate treatment is required of the things 
that determine resolution and how reso- 
lution is measured. Chapter 17 is devoted 
to this subject. Also, relevant information 
about resolution and pixel density can be 
found in the camera reviews on dpreview. 
com and dxomark.com. For the series of 
Canon DSLRs starting with the 10D, the 


measured vertical resolution is proportional 
to the number of pixels (in the vertical direc- 
tion). In fact, the resolved number of lines/ 
mm is about 80% of the pixels/mm. This is 
remarkable, since the observed resolution is 
expected to be reduced by the anti-aliasing 
filter and the averaging effects of the demo- 
saicing step in image processing. Resolution 
is an elusive thing, and, as I show in the next 
chapter, it cannot be adequately expressed 
by a single number. 
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CHAPTER 1 7 


What Is Perceived Image Quality? 


The practical effect of high-resolution technique is less 
that of increasing quality as reducing degradation. 
—JOHN B. WILLIAMS 


Psychophysics: The behavioral study of quantitative 
relations between people's perceptual experiences 
and corresponding physical properties. 
—STEPHEN E. PALMER 


17.1 Introduction 


veryone wants to be able to make 
Ee sharp photographs, and most 

of us immediately recognize whether 
an image is sharp. Also, I have noticed that 
judges in photo competitions often say that 
they are checking for “critical sharpness’ 
But what does that really mean? We have to 
look deeper and define some terms. What 
photographers are really concerned with 
is image quality (IQ), a feature also known 
as image clarity. That is to say, how close 
do the details in the image match those in 
the original scene? The assessment of IQ, 
of course, depends both on the character- 
istics of the image and the perception of an 
observer. This leads to additional descrip- 
tors that are the subject of this chapter. The 
list, I propose, goes as follows: 


Image quality (IQ) = clarity 
Resolution = spatial resolving power 
Visibility = contrast 

Sharpness = acutance 


Our goal is to maximize perceived IQ, 
and our task is to find how the other items 
on the list affect this perception. This is 
important to photographers because we 
need to know how the features of lenses 
and sensors affect our images. We also 
must make decisions about resolution set- 
tings for printing images of various sizes, 
and beyond that there are questions about 
how much resolution is needed for com- 
puter monitors, tablets, digital projectors, 
and HDTV. What we show in this chap- 
ter is that resolution cannot be defined 
satisfactorily by a single number, and, fur- 
thermore, that maximum resolution is not 
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the most important factor in determining 
IQ. 

I will first consider the factors that con- 
tribute to the degradation of images and 
how those factors compound. I will then 
describe in some detail how the perfor- 
mance of cameras can be measured and 
presented. This part is purely technical. The 
image is projected by a lens and captured by 
a sensor. The image may also be processed 
after capture to enhance its appearance. 
I will then consider factors that depend on 
properties of the eye and the wiring of nerve 
cells in the retina and the visual cortex. I will 
show how quantitative information about 
image resolution and contrast can be com- 
bined with factors that influence human 
perception to help us make decisions about 
photographic equipment and its use. Each 
of us is different, of course, and we do not 
see things exactly the same way. Therefore, 
the choices will be personal. 

17.2 Image Blurring and Softness 


/ 


uppose that we photograph a distant 
point of light, perhaps a star or an 
illuminated pinhole. The ideal image 
would have no width at all and might expose 
only a few pixels on the sensor. In the real 
world, lens aberrations, diffraction, and a 
host of other effects contribute to a much 


A 


A 


larger area of exposure. This distribution of 
intensity is called the point spread function 
(PSF). Some point images and their PSFs are 
shown in Figure 17.1. Before we attempt to 
characterize the PSF, it is useful to list some 
major causes of image blurring. Here they 
are in what I believe to be a likely order of 
importance: 


Improper focus 

Camera or subject motion 
Low signal-to-noise ratio 
Diffraction broadening 


Lens aberrations 
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Insufficient resolution of sensor or film 


For some readers the order may be very 
different because of the care they take in 
making images. It is important to note, 
however, that the first three or maybe four 
items on the list depend on actions taken 
by the user; in other words, they are subject 
to user error. 

We know that the PSF will always depend 
on more than one factor, and it is important 
to understand at least qualitatively how the 
various contributions affect the observed 
point spread. Taken separately, each item in 
the list broadens the image in a particular 
way, and the only one that is easily charac- 
terized is diffraction broadening. However, 
for present purposes it is sufficient to 


FIGURE 17.1. Blurred images and the associated point spread functions. 
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approximate each broadening function by 
the well-known bell-shaped error curve, 
also known as the Gaussian function.’ Each 
Gaussian is characterized by a radius, r (see 
Figure 17.1), that I define as the half-width 
at half-height (HWHH). For example, sup- 
pose that an image is broadened by both 
diffraction (radius r,) and lens aberrations 
(radius r,) to produce an observed PSF 
with the radius R. The relationship between 
r, and r, and R in general depends on the 
particular broadening function (lineshape 
function) chosen, but in the case of the 
Gaussian function it is given by: 
Pant, =R (17.1) 
If the PSFs for both diffraction and lens 
aberrations have radii of 10 um, when both 
are present the combined radius is (v10? 
+ 10?) um or about 14.14 um. It is more 
instructive to consider the case where one 
contribution is much greater than the other. 
If the radius for diffraction is 10 um and only 
4 um for aberrations, the combined radius 
is 10.8 um, which is clearly dominated by 
the larger contribution. The lesson is that 
image degradation tends to be dominated 
by one major contributor, and that IQ can- 
not be improved very much by minimizing 
the minor contributors. 
Now we are ina position to consider the 
broadening mechanisms one at a time. Let’s 
take the list from the top. 


17.3 Focus 


odern lenses usually autofocus, 
but that does not necessarily 
mean that the desired objects will 


be in focus. The user may have to set a focus 
point or first focus and then recompose, 
depending on the camera. Just as important 
is the depth-of-field (DoF) and the way it 


depends on camera position and settings. 
As a rough rule of thumb, DoF depends 
directly on the F-number (N) and inversely 
on the magnification (m) squared, i.e. DoF 
is proportional to N/m? (see Chapter 12 and 
Appendix J). This means it is very difficult 
to get nearby objects, where m is larger, 
in focus at the same time. Photographs 
of the faces of cats often turn out better than 
the faces of long-nosed dogs because of the 
difficulty of getting both the dog’s eyes and 
nose tip in good focus. 


17.4 Camera Motion 


ere again modern technology 
comes to the rescue with image 
stabilization built into lenses or 


camera bodies. This helps, but does not 
completely solve the problem. It simply 
means that we can get away with somewhat 
slower shutter speeds, perhaps two or three 
stops, when handholding a camera, but it 
does nothing at all for subjects in motion. In 
the absence of image stabilization the gen- 
eral rule is that the shutter speed in seconds 
for 35mm film and FF sensors should be at 
least as short as one divided by the focal 
length of the lens measured in mm; in other 
words [1 mm/f(mm)] sec. For cameras with 
crop factors, the denominator must be 
multiplied by the crop factor. For example a 
Canon 80D camera (crop factor = 1.6) with 
a 100 mm lens should have a shutter speed 
at least as fast as (1/160) sec for handheld 
photographs. 

Serious photographers, who want to be 
able to print big enlargements, use a sturdy 
tripod along with a cable/remote shutter 
release or self-timer. When using single 
lens reflex (SLR) cameras with moving mir- 
rors and mechanical shutters, they use the 
mirror lockup feature and when possible 
take precautions to minimize the effects 
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of shutter vibrations. Those who attempt 
astrophotography know how difficult it is to 
get high-resolution images of the moon and 
planets even when using sturdy tripods and 
good techniques. It should be noted that 
the PSF is not adequate for describing the 
effects of camera or subject motion, which 
may smear an image in one direction so 
that points of light become lines. There are 
image processing tools that can correct, to 
some extent, for unidirectional motion blur. 


17.5 Signal-to-Noise Ratios 


luminance or color from pixel to pixel 

will be severe enough to affect image 
quality. With small point-and-shoot cam- 
eras this is, in fact, likely when the higher 
ranges of ISO sensitivity are selected. With 
any digital camera, noise can become a 
problem with underexposed images, espe- 
cially when there is an attempt to brighten 
the dark areas with image processing tools. 
The problem is that the number of elec- 
trons captured by some pixels in the sensor 
is so small that significant fluctuations in 
numbers are unavoidable. In order to mini- 
mize this problem, photographers should 
avoid high ISO numbers and underexposed 
images. For those who use histograms on 
their digital cameras to monitor exposure, 
this means “exposing to the right” so that 
the intensity distribution is shifted as far to 
the right as possible without clipping at the 
right end. The overall exposure can then be 
corrected with image processing software. 

It is also possible to improve the signal- 
to-noise ratio through signal averaging. 
Here we take advantage of the fact that the 
signal in an image is the sum of the mean 
signal and the randomly varying noise. If 
images are captured repetitively n times 
with the same exposure, aligned, and added 


E is possible that intensity fluctuations in 


in memory, the accumulated value of the 
average signal will increase n-fold while 
the noise will only increase as Vn. Therefore, 
the S/N ratio will also increase as Vn. For 
example, if an image with S/N equal to 10 is 
accumulated 9 times, the resulting S/N ratio 
will equal = 10 - V9 = 30. 

This procedure has been implemented 
in some Canon cameras as the “handheld 
night scene” mode. It is also the basis of the 
HDR+ mode in Google smartphones where 
the images are exposed for the brightest 
areas, and the dark areas are brightened as 
required in the signal processing step. This 
works because of the accompanying noise 
reduction. Furthermore, the underexposing 
of images in HDR+ permits higher shutter 
speeds and shorter accumulation times 
than is possible with conventional HDR 
methods where the brighter images require 
more exposure. 


17.6 Diffraction 


he diffraction phenomenon was 

discussed in Chapters 5 and 11. 

Basically, when a point of light is 
imaged using a lens with an F-number 
equal to N, the result is a pattern of light on 
the sensor that consists of a bright center 
spot (Airy disk) encircled by a dark ring, a 
faint bright ring, and so on becoming ever 
weaker. The diameter of the first dark ring 
is (1.35 N) um for green light (550 nm), and 
this is a big deal since the pixel width in 
some small cameras is only about 1.5 um. So 
there is an incentive to keep N small, which 
is why compact cameras limit N to //8 or 
even smaller. In some cameras at maxi- 
mum telephoto zoom where small values of 
N are not available, the resolution is always 
limited by diffraction. Of course, there is a 
trade-off between diffraction and DoF. This 
problem was discussed in Chapter 11, and 
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it was suggested that significant diffraction 
effects can be avoided by making sure that 
N is less than the diagonal dimension of the 
sensor in mm divided by two. For example, 
a small camera with a 1/2.5 in. sensor where 
the diagonal dimension is 7.18 mm should 
always be used with N < 7.18/2 or 3.6. This 
computation assumes that prints will be 11 
in. x 14 in. or less and will be viewed from 
about 13 in. 


17.7 Lens Performance 


inally, we come to factors that are 
Esos: the control of the user. Of 

course, a lens may perform better at 
some F-number settings and some focal 
lengths, but still our options are restricted. 
So how can we evaluate the performance 
of a lens? Basically, we need to know how 
completely the image replicates the details 
in the original screen. Traditionally, a lens- 
resolution chart with black and white line 
pairs (lp/mm) or sinusoidal patterns (cycles/ 
mm) would be photographed to determine 
the maximum number of lines pairs/mm or 
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cycles/mm that could be distinguished on 
film or a sensor. This idea is illustrated in 
Figure 17.2 with simulated images of a sinu- 
soidal intensity pattern where the frequency 
in cycles/mm increases to the right. In part 
(a), the lens introduces a PSF witha standard 
deviation (0) of 0.03 mm that corresponds 
to a radius of 1.1770. This broadening effect 
causes the more closely spaced lines on 
the right to overlap so that the contrast is 
reduced. In part (b), the standard deviation 
of the PSF is smaller and the higher frequen- 
cies are less attenuated. The important point 
is that the highest observed resolution is 
always associated with the lowest contrast. 
We see immediately that a single number 
is not very useful in specifying resolution. 
We need to know what contrast or visibility 
is associated with each resolution. The con- 
ventional way to display this information 
is to plot the contrast versus the resolu- 
tion (cycles/mm for sinusoidal patterns or 
Ip/mm for square wave patterns) for a lens 
at a particular F-number. This defines one 
form of the modulation transfer function 
(MTE). 


PSF 
o = 0.03mm 


PSF 
o = 0.0lmm 


0 02 


FIGURE 17.2. The effect of PSFs with different standard deviations on the contrast of test pattern 


images for (a) o = 0.03mm and (b) o = 0.01mm. 
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MTFs are a very important means of 
characterizing lenses, and photographers 
need to understand them. For the red inten- 
sity plots in Figure 17.2 one can compute 
the contrast at each frequency by measuring 
the maximum and minimum intensity and 
taking the ratio, J, —1£..)/(UL,. +1,..). This 
is shown in part (a) by the intensity meas- 
urements near the 3 mm position where the 
frequency is 6 to 7 cycles/mm. The same 
information is contained in the shape and 
width of the PSF as in the contrast versus 
frequency measurements. This is easy to 
understand because the PSF is a signature 
of the ranges of spatial frequencies that the 
lens system can pass. A sharp, narrow PSF 
indicates that high frequencies are present, 
and a mathematical (Fourier) transforma- 
tion can quantify the contributions from all 
of the frequency components.’ The analysis 
of lens performance makes use of the fact 
that the PSF, the line spread function (LSF), 
and the MTF are all related.* 

Consider a good lens with very small 
aberrations so that the images at f/8 and 
f/22 are blurred primarily by diffraction. 


Contrast 
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The MTFs that characterize the lens at these 
apertures are shown in Figure 17.3. All MTF 
curves begin at 100% contrast at low fre- 
quencies and eventually reach zero contrast 
at high frequencies. MTF curves exhibit a 
wide variety of shapes and so a given point 
such as 60% contrast at 100 cycles/mm does 
not uniquely define a MTF curve. The area 
under a curve turns out to be a better indi- 
cator of IQ than the maximum frequency 
reached.* 

An MTF clearly contains a lot of informa- 
tion about resolution and visibility, but that 
is just the beginning. In principle the MTF 
must be measured at not only a wide range 
of F-numbers and focal lengths for zoom 
lenses but also in two orientations at repre- 
sentative points in the image plane. At the 
center of the image these patterns can have 
either vertical or horizontal orientation. 
The difference, which results from astigma- 
tism, is usually small and the average can be 
taken. Significant differences are expected 
to appear when one moves away from the 
center (the optical axis). Test patterns are 
usually oriented in the radial direction from 
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FIGURE 17.3. MTF curves for a diffraction limited lens at f/8 and f/22. 
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FIGURE 17.4. A sensor showing test points (red) and patterns in the sagittal and meridional 
directions. The length of the arrow is 22 mm for a FF DSLR. 


the center (meridional plane) or perpen- 
dicular to that direction (sagittal plane). 
Similarly, the line and point spread func- 
tions can be analyzed in those directions. 

The excessive amount of information in 
numerous MTF plots provides the moti- 
vation for a simplified type of display, also 
labeled MTF. The idea is to select a single 
frequency value, say 30 cycles/mm, and to 
plot the measured contrast as a function 
of distance from the center of the image 
for both meridional (tangential) and sagit- 
tal orientations. Figure 17.4 illustrates test 
points in an FF image. 

Some lens manufacturers use 10 lp/ 
mm and 30 lp/mm patterns to empha- 
size contrast and resolution, respectively. 
Also, lenses are tested at full aperture and 
stopped down. A typical set of MTFs is 
shown in Figure 17.5 for a hypothetical high 
quality 50 mm lens with a maximum aper- 
ture setting of f/2.0. Keep in mind that the 
abscissa specifies the distance of the meas- 
ured point on the diagonal from the center 
of the sensor. 


The distance from the center to the cor- 
ner is approximately 22 mm for the full 
frame sensor and only 14 mm for a sensor 
with a crop factor of 1.6. As rules of thumb, 
10 lp/mm curves with contrast above 0.8 
and 30 lp/mm curves above 0.6 represent 
high quality images. It is much more dif- 
ficult to construct wide-angle lenses that 
fall in this range than is the case with long 
focal length lenses. Therefore, only lenses in 
the same focal length range should be com- 
pared. “Contrasty” lenses are indicated by 
an elevated level of the 10 lp/mm curves, 
while the 30 lp/mm curves reveal resolving 
power. Also, the most pleasing representa- 
tion of the out of focus points, i.e. bokeh, 
occurs when the sagittal and meridional 
curves are close together. Most lenses are 
not at their best at full aperture. It is quite 
common to find that lenses perform better 
a couple of stops smaller than the maximum 
aperture. 

It is necessary to make two additional 
points. First, the line pair frequencies shown 
above are reasonable for the enlargement 
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FIGURE 17.5. MTF for a high quality 50 mm f/2.0 lens wide open (f/2.0). Meridional line (M) is 


dashed and sagittal line (S) is solid. 


factors commonly used with 35 mm cameras 
and FF DSLRs. When smaller sensors with 
larger crop factors are used, the required 
enlargement factors will also be larger. For 
example, from the FF, 24 mm x 36 mm, 
sensor to an 8 in. x 10 in. print requires an 
enlargement factor of 8, while the 4/3 in., 
13.5 mm x 18 mm, sensor used by Olympus 
and Panasonic would require a factor of 15. 
Therefore, it makes sense for Olympus and 
Panasonic to select 20 lp/mm and 60 lp/mm 
for MTF plots. Also, remember that these 
MTF plots only take into account the per- 
formance of the lens. In order to describe 
the results when the lens is used with a 
particular camera, the contribution of the 
sensor must also be considered. 


17.8 Sensor Resolution 


he operation of digital sensors was 
considered in some detail in Chapter 


16. A sensor samples images by 


means of a periodic array of detectors. The 
analogy between the detectors and grain in 
film breaks down here because grain has a 
random spatial distribution. The implica- 
tions of this difference become clear when 
we recall that all images can be resolved into 
sets of periodic patterns with different fre- 
quencies. Consider Figure 17.6, where the 
black and white image (bar) at the top is 
represented by the solid black line imme- 
diately under it in the graph. This is part 
of a repetitive sequence of black and white 
bars. I performed a (Fourier) reconstruction 
of this shape by adding together sinusoidal 
functions with ever increasing frequencies. 
The bottom curve contains one frequency 
component (1 cycle/mm), the curve labeled 
2 contains two, and so on up to 20. The 
green curve labeled 20 is the best I can do 
when the highest frequency component 
is only 20 cycles/mm. The 20 cycle/mm 
component by itself is represented by the 
magenta curve. 
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Amplitude 


FIGURE 17.6. Fourier series analysis of a square wave image. 


It is clear that an accurate represen- 
tation of this image with its sharp edges 
would require much higher frequencies. 
So we need to know the highest frequency 
that the sensor can faithfully record. The 
answer comes from the Nyquist theorem, 
which tells us that at least two samples are 
required per cycle (or line pair) in order to 
establish a frequency component. In imag- 
ing applications the length of a cycle, its 
wavelength A, is expressed in mm and the 
spatial frequency is specified as (1/A) mm”. 
Therefore, according to the Nyquist theorem 
the spatial frequency (1/A) mm” requires a 
sampling rate of at least (2/A) mm”. This 
sampling frequency is called the Nyquist 
limit. The components with frequencies that 
are too high for this sampling rate do not go 
away, they are just misinterpreted as lower- 
frequency components and added to the 
image. Figure 17.7 shows how this happens. 

The true frequency component (magenta) 
is sampled at too low a rate, shown by the 


dots, and is misinterpreted as the blue curve. 
This effect is called aliasing, and the only way 
to avoid it is to remove frequencies beyond 
the Nyquist limit from the image by means 
of a low pass or anti-aliasing (AA) filter. This 
intentional blurring of images was introduced 
in Chapter 16. The idea is to make sure that 
the image projected onto the sensor does not 
have any frequency components higher than 
one half the number of pixels/mm. So this 
is only a problem for lenses with resolving 
powers that exceed the Nyquist limit of the 
sensors, and even with these lenses the reso- 
lution may already be reduced by aperture 
dependent diffraction. Every year the pixel 
count in sensors increases so that at some 
point sensors will out resolve all lenses. Many 
camera companies have already concluded 
that AA filters are not worthwhile, and they 
leave out the AA filter on some cameras all 
together or make it optional. Some other 
manufacturers are sticking with AA filters 
for the current generation of cameras. 
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FIGURE 17.7. Illustration of the Nyquist theorem. The true frequency is shown in magenta, but 
inadequate sampling incorrectly identifies the lower-frequency blue curve. 


Our concern with the Nyquist limit 
should not leave the impression that all 
frequency components below that limit 
are treated equally. In fact, a sensor causes 
some low pass filtering because of its geom- 
etry, the efficiency of the pixel lenses, and 
the diffusion of charge carriers. These fac- 
tors determine the MTF (contrast versus 
frequency) for the sensor, and the MTF for 
the complete photographic system is just 
the product of the individual MTFs. An 
equivalent description is to say that the lens 
and the sensor each contribute to the over- 
all system PSF. 

So where does that leave us with regard 
to sensor resolution? A monochrome 
sensor with no CFA can perhaps be rep- 
resented by a PSF with a width equal to a 
couple of pixels. An AA filter may or may 
not broaden the image by a couple of pix- 
els, and the color filter array will insure 
that the pixels associated with R, B, or G 
green light will be separated by a pixel or 
so. So let’s take about three pixel widths as 
the minimum PSF associated with a sensor. 
In Chapter 11 I suggested that the circle of 


confusion (CoC), for computing focus that 
is “good enough,’ could be defined as the 
diagonal dimension of the sensor divided 
by 1500. This seems reasonable for prints 
that are viewed at a distance equal to their 
diagonal dimension. If all prints are viewed 
from a standard viewing distance of about 
13 in., regardless of their size, then the divi- 
sor should probably be scaled up by the 
ratio of the print width to that of an 8 in. 
x 10 in. print. For example, a 16 in. x 20 in. 
print would require a CoC equal to sensor 
diagonal divided by 3000. While this is, in 
principle, reasonable, the practical lower 
limit for the CoC is about three pixel widths. 
With the 12.1 megapixel, 1/1.7 in. sensor 
of the Canon G16 camera, division of the 
diagonal by 1500 already yields only three 
pixel widths; and it doesn’t make sense to 
increase the divisor. Furthermore, diffrac- 
tion broadening already exceeds this CoC at 
F-numbers greater than 4.8, so there is not 
much room to maneuver. 
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17.9 Perceived Image Quality 


o far I have considered objective meas- 
S ures of resolution and contrast. Now we 

come to the really important part. How 
does the image look? Attempts to quantify 
image quality are, of course, not new. First, 
there is the question of sharpness as opposed 
to resolution. Already in the 1960s acutance 
was being advanced as an “objective correlate 
of sharpness.’ Early definitions of acutance 
were based on the steepness of edges or how 
quickly the intensity or the density in an 
image changes with distance. One measure of 
acutance was the square of the slope of a plot 
of the image density across an edge. The next 
step was taken by E. M. Crane of the Kodak 
Research Laboratories, who reported an 
objective method for rating picture sharpness 
that had been “tested in the mill of widely var- 
ied experience” His conclusion was that the 
perceived sharpness is proportional to the 
area under the MTF squared for the system 
when the MTF is plotted on a logarithmic 
frequency axis.’ I will return to this idea in 
Section 17.12, on subjective image quality. 
At this point I turn to the popular notion of 
sharpening an image. 


17.10 Sharpening by Means of 
Image Processing 


hus far, every section in this chap- 
ter has been concerned with image 


degradation. What about improving 


Ideal image PSF 


EMAC 


the image with processing? The image 
from a camera can be thought of as an 
ideal image in which every point has been 
replaced with a PSF, i.e. a fuzzy dot, having 
the intensity and color of the original point. 
All the PSFs are then added up to produce 
the final image. This operation is known 
as the convolution of the ideal image with 
the broadening function. A trivial illustra- 
tion of this process is shown in Figure 17.8. 
The natural question is whether it is pos- 
sible to deconvolute a real image to obtain 
a better approximation to the ideal image. 
A perfect deconvolution is not possible; 
but, if the shape of the broadening function 
(PSF) is known, the deconvolution can be 
approximated. Deconvolution is, in fact, 
the basis of some of the “smart sharpening” 
programs. Focus Magic is an example of a 
deconvolution based program. 

Other sharpening programs work directly 
to enhance edges in the image. This can be 
done either by means of high pass filters 
or by checking the brightness of neighbor- 
ing pixels. Edges are enhanced by making 
the pixels on the bright side even brighter 
and those on the darker side even darker. A 
blurred but sharpened edge between dark 
gray and light gray is shown in the image in 
Figure 17.9. The brightness plots show the 
original image (red dashed line) and two lev- 
els of sharpening with high pass filters (black 
solid line and blue dashed line). It is easy to 
overdo this kind of sharpening and to gener- 
ate obvious halos around the edges. 


Real image 


FIGURE 17.8. Convolution of an ideal image with a PSF to give a real image. 
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FIGURE 17.9. A fuzzy edge (red line) with two levels of sharpening by means of a high pass filter. 


A recommended workflow for sharpen- 
ing is first to enlarge the input image from 
the camera to 100% (one pixel on the com- 
puter screen equals one pixel in the camera) 
and to apply sufficient sharpening to coun- 
teract the broadening effects of the AA 
filter and the characteristics of the sensor. 
Then, after the image has been corrected 
and the final print size has been selected, 
the image is resized and an appropriate 
output sharpening can be applied. Here it 
is sometimes advantageous to “res” up the 
image to at least 350 pixels/in. before apply- 
ing the final sharpening. It is convenient 
to use sharpening filters built into photo- 
processing programs such as Photoshop 
and Lightroom, but there are also sharp- 
ening programs from various vendors as 
either plugins or stand-alones. Numerous 
programs including the following are availa- 
ble online: PhotoKit Sharpener, FocalBlade, 
Nik Sharpener Pro, and Topaz Detail. 


17.11 Contrast Sensitivity Function 


ow we face the question of what 
determines our subjective assess- 
ment of image quality. Crane’s 


finding that sharpness is determined more 
by the area under the MTF curve than by 
the highest resolution reached is a tip-off. 
The area measurement tends to emphasize 
the lower frequencies and higher contrasts. 
Also, physiologists have reported that 
human resolving ability depends strongly 
on contrast.” Subjects were asked whether 
they could detect a grating pattern at lower 
and lower contrast levels. The purpose was 
to determine a contrast threshold below 
which no pattern could be discerned. The 
surprising conclusion was that the thresh- 
old level depends strongly on the frequency 
of the grating, and the lowest threshold is 
in the vicinity of 6 cycles per degree. The 
contrast sensitivity function (CSF) is defined 
as the inverse of the threshold, so this func- 
tion has a peak in the range of 3 to 12 cycles/ 
degree and falls off both at higher and lower 
frequencies. At a normal viewing distance 
of 34 cm, or slightly more than 13 in., this 
amounts to maximum sensitivity range 
from 0.5 cycles/mm to 2.0 cycles/mm. 
Figure 17.10 provides a visual confir- 
mation of the CSF. The grating frequency 
increases to the right on a logarithmic 
scale and contrast decreases in the vertical 
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FIGURE 17.10. A plot of contrast (increasing toward the bottom) versus frequency on a logarith- 


mic axis. 


direction, and most people can more eas- 
ily resolve full lines near the center than at 
the sides. Of course, the angular spread of 
this pattern and the viewing effect depend 
on the viewing distance. Also, more recent 
research shows that the sensitivity peak 
shifts to slightly higher frequencies at 
higher illumination levels. The MTF for the 
eye falls off at high frequencies because of 
aberrations and diffraction, and the effective 
MTF for the entire human visual system is 
expected to be the product of MTF for the 
eye and the CSF. 

These results imply that the cells in the 
striate cortex are tuned to different spatial 
frequencies. This turns out to be the case. 
Ingenious experiments with gratings con- 
taining various shapes, such as line pairs 
instead of sinusoidal patterns, show that the 
(Fourier) frequency components of patterns 
can be distinguished by the human visual 
system.? For example, the square wave in 
Figure 17.6 contains a fundamental fre- 
quency (one term) and higher harmonics. 
At lower contrasts only the fundamental 
exceeds the threshold, and the pattern is 
indistinguishable from a sinusoidal pattern. 
At contrasts above the threshold for the next 


harmonic, a difference can be distinguished 
between line pairs and sine curves when 
the two patterns are repetitively switched 
on a computer screen. Of course, the stri- 
ate cells have limited spatial extent, and 
they can collect only small sections of sine 
curves containing small numbers of cycles. 
Apparently, analysis is possible piece-wise 
with small patches of sine curves that fade 
out with distance. Such oscillatory pat- 
terns with bell-shaped envelopes have been 
described mathematically and are known as 
wavelets.*” 


17.12 Subjective Quality Factor 


ur initial goal was to correlate 

perceived image quality with 

measurable properties of images. 
In 1972 Edward Granger, a senior scien- 
tist at Eastman Kodak, proposed an optical 
merit function that goes a long way toward 
accomplishing this goal. His work combined 
Crane’s insight about the importance of the 
area under the MTF curve with research on 
the contrast sensitivity of the human visual 
system. What he did was to use CSF data 
to determine the most relevant part of the 
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MTF for images of a certain size. Let us 
return to the MTF in Figure 17.3 and plot 
it with a logarithmic frequency axis. The 
resulting curves are shown in Figure 17.11. 

Granger defined the subjective quality 
factor (SQF) to be proportional to the area 
under a MTF curve (on a logarithmetic fre- 
quency axis) in the frequency range that 
corresponds to 0.5 to 2.0 cycles/mm on a 
print viewed from 34 cm.51 Consider, for 
example, 4 in. x 6 in. and 20 in. x 30 in. 
prints. A FF sensor (1 in. x 1.5 in.) image 
would have to be enlarged by factors of 4 
and 20, respectively, to achieve these print 
sizes, and the required frequency ranges on 
the sensor would be 2-8 cycles/mm and 
10-40 cycles/mm. For the 4 in. x 6 in. print 
the SQF would be determined by the area 
between the red lines under the appropri- 
ate MTF curve in Figure 17.11. Similarly, 
the SQF for the 20 in. x 30 in. print would 
depend on the area between the blue lines 
under the selected MTF curve. The con- 
clusion is that both the f/8 and //22 curves 
provide similar SQFs for 4 in. x 6 in. prints 


viewed at 13 in. However, the subjective 
quality for the 20 in. x 30 in. print at 13 in. 
is lower at //22 than at f/8 and also much 
lower than that for the 4 in. x 6 in. print 
at either f/8 or f/22. This type of quality 
assessment works in the real world because 
high-frequency components with low con- 
trast are actually not seen. 

The beauty of the SQF is that it pro- 
vides a single number for evaluating a lens 
or optical system. Of course, the work is 
not quite over because we know that the 
MTF curves depend on where in the image 
frame they are measured and also on the 
orientation of the test pattern. The SQF 
system described here is used as the basis 
of the lens evaluations reported by Popular 
Photography.**"1 They are able to pro- 
vide a single number to describe the SQF 
for the complete image frame at each print 
size and F-number by averaging measure- 
ments at image locations in the following 
way: center (50% weight), 50% of the way to 
a corner (30% weight), and 80% of the way 
to a corner (20% weight). The test points are 
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FIGURE 17.11. 
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MTF for a diffraction-limited lens. Regions: 2-8 cycles/mm (red) and 10-40 cycles/mm 
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shown in Figure 17.4. The SQF numbers are 
adjusted to a 100 point scale and then are 
assigned letter grades (and colors) from A 
through F in 10 point steps. This is a rea- 
sonable procedure, but the user is left with 
no information about how quality is distrib- 
uted across the image. For example, a lens 
with a rating of 86 for a given print size and 
F-number may have achieved 86 at all test 
sites or the scores may have been 100, 80, 
and 60, to give 100 x 0.5 + 80 x 0.3 + 60 x 0.2 
= 86. The user may prefer center sharpness 
and a fall off in quality toward the corners 
rather than overall softness. (See the Further 
Reading section for more on SQF, including 
the excellent review article by Larry White 
on methods used by Popular Photography.) 

The only way to evaluate a lens, with- 
out hands-on experience, is to use several 
sources of information. In addition to the 
SQF reports and MTF curves reported 
by vendors, several websites now publish 
detailed tests of popular lenses. Perhaps 
the largest collection can be found at www. 
photozone.de. Also, www.dprereview.com 
is expanding its coverage of lenses; and 
interesting lenses are often discussed on 
www.luminous-landscape.com. In addition, 
Roger Cicala offers excellent technical anal- 
yses and reviews at https://www.lensrentals. 
com/blog. Even this level of diligence does 
not insure success because there is still 
sample-to-sample variation in lens perfor- 
mance. New lenses should be tested and, 
if found to be deficient, returned to the 
vendor; although there are some cases, e.g. 
backfocus error, where recalibration may fix 
the problem. 


17.13 A Caveat 


that everything in photography involves 


¡E this chapter by reminding readers 
compromises. Now that images are 


digital and everyone can blow them up to 
100% and beyond on computer screens, it 
is easy to find defects in any lens or cam- 
era system. We all want large zoom ranges 
and large apertures with high image quality 
in any size print we choose to make. With 
present technology and materials this will 
not happen, although we can get a bit closer 
at high price levels. The new generation of 
super-zoom consumer lenses for DSLRs 
(1.5-1.6 crop factor) illustrates the point. 
There is a big market for all-purpose, super- 
zoom lenses, and the major vendors have all 
attempted to capture it. I am talking about 
the 18-300 mm zoom lenses from Canon, 
Nikon, and Sigma as well as the 16-300 
mm lens from Tamron. Each lens involves 
slightly different compromises, but the limi- 
tations oftechnology and the laws of nature 
are evident in the final products. 

First, the maximum apertures are all 
modest and they shrink as the focal length 
increases. We find f/3.5-f/5.6 for Canon 
and Nikon and f/3.5-f/6.3 for Sigma and 
Tamron. We also observe the impossibil- 
ity of uniformly high optical performance 
over the 10X and greater zoom ranges. With 
lenses having zoom ranges up to 4X, manu- 
facturers do rather well; but that is about 
the limit. The present collection of lenses 
tends to show barrel distortion, vignetting, 
soft corners, and chromatic aberrations at 
18 mm. As the focal length increases, the 
resolution and distortion change in different 
ways for each of the lenses. Typically, reso- 
lution is best in the middle range, but one 
of the Sigma lenses shows different compro- 
mises. Here the extreme focal lengths are 
sweet spots for resolution, while it has been 
reported that resolution at 35 mm is not as 
good. There is also the problem that near 
focus is usually achieved at least in part by 
focal length reduction or “focus breathing.” 
For example, photographs of nearby birds at 
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the 300 mm setting with a super-zoom lens 
may be disappointing because the effective 
focal length turns out to be much less than 
300 mm. 

Overall these lenses are optical marvels 
if one is not too critical. In fact, recommen- 
dations often hinge on non-optical factors 
such as focusing accuracy and efficiency 
of the image stabilization system. Even if 
optical and mechanical parts are good, 
the potential buyer may be put off by the 
weight, the price, or the length of the war- 
ranty. Photographers must decide what 
compromises they are willing to make. They 
should also remind themselves that defects 
in photographs can seldom be blamed on 
lens quality. Good photographic techniques 
and an appropriate lens hood to minimize 
lens flare can work wonders with any lens. 
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CHAPTER 18 


The Creation and Appreciation 
of Art in Photography 


What people find beautiful is not arbitrary or random but 
has evolved over millions of years of hominid sensory, 
perceptual, and cognitive development. 


—MICHAEL S. GAZZANIGA 


Everything the visual system does is based 
on (such) educated guesswork. 


—V. S. RAMACHANDRAN AND S. BLAKESLEE 


18.1 What Is a Good Photograph? 


o most casual camera users, pho- 

| tography provides documentation 
of people and events. Photographs 

“save” memories and are usually judged by 
their content. Did the photograph capture a 
likeness, was there good timing, and so on? 
Family and friends glance at photographs 
and say “that is a good one” or maybe “Id like 
a copy.’ When the photographer has better, 
more expensive, equipment, there is a natu- 
ral tendency to expect better photographs. 
When more is expected, there will be more 
disappointments. The photographer now 
wants good exposures, sharp images, and 
natural colors. These are technical require- 
ments, and with some study and experience 


the photographer can learn what is possible 
with the available equipment. 

At the next level we encounter amateur 
photographers who know about lenses, 
the importance of using tripods, and many 
technical details. They have researched 
cameras and have equipped themselves to 
make technically perfect images. Now the 
standards are higher, and the photographer 
wants approval from more than family and 
friends. First, there are other photogra- 
phers, who are also potential competitors. 
Then there are camera clubs with photo 
competitions and experienced judges. This 
can be painful to the beginner because a 
judge might be unmoved by a favorite pho- 
tograph or may dismiss a photograph out 
of hand because some rule of composition 
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has been violated. Also, two judges may 
have very different reactions to the same 
photograph. 

At this point, an amateur photographer 
may simply reject photo competitions and 
enjoy his or her photography as it is. This 
is certainly a valid option and is the one 
chosen by most amateurs. There are others, 
however, who enjoy competition and who 
desire feedback from other photographers. 
Photography is a big thing for these indi- 
viduals, and they are willing to devote a lot 
of time and money to improving their pho- 
tographic skills. They read books, attend 
workshops, and go on photo shoots with 
the aim of learning how to make better 
photographs. They are serious amateurs, 
and they may aspire to become professional 
photographers. 

With the serious amateur in mind, I 
return to the role of judges. Their work is 
certainly subjective, but observations of 
many judges over time reveal some consist- 
ency. (I am ignoring truly bad judges who 
exhibit unreasonable biases and a lack of 
expertise.) Some judges describe the infor- 
mal guidelines they use and are willing to 
critique in detail images that they have 
judged. Here is a sample set of guidelines: 


Technical perfection (exposure, color, 
critical sharpness) 


Composition (pleasing layout, may involve 
conventional rules of composition) 


Impact (eye-catching, novel, etc.) 


These points may be given weights, such 
as 30% for technical perfection, 30% for 
composition, and 40% for impact. What is 
clear is that the judge is dealing with vis- 
ual art, and the process is essentially the 
same regardless of how the art was created. 
This is more true now than ever before for 
photographic competitions because of the 


widespread manipulation of digital images. 
Nature photography contests may impose 
restrictions, but some level of correction 
and adjustment is usually permitted even 
there. 

The bottom line is that we are dealing 
with visual art, and judgments are subjec- 
tive, but not completely so. There are rules 
of composition, but they are nothing more 
than lists of features that either appear in 
or are missing from images that appeal to 
experienced observers. For example, an 
object in the exact center of an image often 
makes for a boring picture. One “rule” sug- 
gests that an object should be placed one 
third of the way from an edge of the frame, 
but there are many exceptions. I will return 
to the rules later, but here I am concerned 
with more fundamental questions. What 
is art after all, and why are certain images 
appealing? These are the questions that I 
attempt to address, but by no means settle, 
in this chapter. 

Are the creation and appreciation of art 
valid topics for a science book? Absolutely! 
It is all about the brain. (In order to avoid 
confusion with terminology, let me distin- 
guish between brain and mind. The brain 
is a physical object, and the mind is what 
goes on in the brain.) The real question is 
whether the neurosciences have advanced 
to the point that they can help us under- 
stand creation and evaluation of visual art. 
This is an exciting area of research, and a lot 
of related work is going on. In Chapter 15, 
on the perception of color, we hada glimpse 
of what is involved. A number of distinct 
areas of the brain work simultaneously on 
different aspects of a visual image, assign- 
ing color, detecting motion, recognizing 
faces, etc., and by an unknown mechanism 
the conscious mind experiences and under- 
stands a consistent, and even beautiful, 
image. 
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This chapter turns out to be an intro- 
duction to visual perception and art. I have 
made use of numerous recent publications 
on cognition and art, including Margaret 
Livingstone’s beautiful book, Vision and 
Art.’ Also, Robert L. Solso’s books on cog- 
nition, art, and the conscious brain”? and 
Semir Zeki’s book, Inner Vision* deserve 
special mention. More recently, Michael S. 
Gazzaniga has addressed art and the human 
brain in his book, Human.’ Of course, 
the bible of human visual perception is the 
wonderful book, Vision Science, by Stephen 
Palmer.’ Finally, I note that this area of sci- 
ence has strong overlap with the exploding 
area of Computational Photography.”* 


18.2 Art and Consciousness 


o, what is art? The definition of art is 

controversial, since there is the danger 

that any definition will be too restric- 
tive. That may be, but this is my provisional 
definition of visual art: Visual art is an 
expression of human creativity that causes 
pleasure, and by the aesthetic judgment of 
the human mind it is somehow stimulating 
and often experienced as beautiful. If this is 
too general for the photographer, just think 
of two-dimensional, framed images that 
contain either patterns or representations 
of the world. Other definitions of art will 
be stated or implied in later sections of this 
chapter, but I do not accept the idea that all 
art is bound to culture. I am concerned with 
more universal, cross-cultural aspects of art. 
Also, my concept of art is closely connected 
with beauty, which indeed is universal. 
Steven Pinker points out that culture, fash- 
ion, and psychology of status (imparted by 
one’s judgment and also ownership of art) 
come into play in addition to the psychology 
of aesthetics when judging art.? I suspect 
that all of this is true, but I will attempt 


to focus on works that are aesthetically 
pleasing and instructive. 

Art is conceived by and for the human 
mind. But why does our mind have this 
capability and inclination? It is reasonable 
to assume that the brain evolved for sur- 
vival value, and sight evolved because of 
the knowledge it gives us about the world. 
We can speculate that visual capabilities 
evolved because of particular needs. For 
example, color assignment and color con- 
stancy aid in the selection of ripe fruits. 
Robert Solso emphasizes that the brain 
evolved to facilitate the primal functions: 
hunting, killing, eating, and having sex.”° 
These activities are, of course, not limited 
to humans. The ability to create and appre- 
ciate art is uniquely human, however, and it 
may simply be a by-product of the evolution 
of the complex and adaptive human brain.” 
There are strong arguments that art plays 
a functional role in the development and 
organization of the brain, making humans 
more effective and more competitive in a 
variety of ways. 

By 100,000 years ago the brain of homo 
sapiens had reached, more or less, its mod- 
ern capacity. Undoubtedly, people from 
that period and later produced decorations 
and carvings, but the earliest paintings 
that have been found date from about 
35,000 BCE. These are cave paintings in 
the caves of southern France and in Italy, 
as shown in Figure 18.1. Among the old- 
est are those found in the Grotte Chauvet, 
which contains about 300 murals. Less well- 
preserved, but possibly older, images have 
been found in the Fumane Cave northwest 
of Verona.'”!! More recent finds of rock and 
cave art have been reported in Indonesia 
and Australia. The Paleolithic paintings, 
mainly of animals, sometimes show depth 
and are executed in vivid colors. It was 
Solso’s thesis that the human brain evolved 
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FIGURE 18.1. An example of murals from Lascaux, France (c. 17,000 BCE). 


to the level of complexity and adaptability 
that supported a high level of consciousness 
(awareness) and that, in turn, enabled art, or 
conversely that art reveals for us the appear- 
ance of conscious minds in the Stone Age. 

It all comes back to the capabilities of the 
human mind. We are blessed with a compact 
brain that nonetheless provides amazing 
power through multichannel/multiproc- 
essing. This is computer talk which may be 
inappropriate. The brain is a machine that 
is doing some kind of information process- 
ing, but it may not be a computer. What 
we know is that it contains 100 billion neu- 
rons (nerve cells) and that each neuron is 
connected across synapses to as many as 
100,000 other neurons. Furthermore, the 
connections are constantly changing in 
response to changing stimuli and the his- 
tory of stimuli. Consequently, a mind will 
not respond exactly the same way to an 
identical stimulus the second time. Those 
in the artificial intelligence (AI) community, 
who hope to simulate the human mind, face 
a daunting task.” 

Consciousness is the feature of the 
mind most closely associated with art, but 
it is also the least understood feature. We 
can say a lot about the characteristics of 
consciousness, but we are far from under- 
standing what it really is, or how it comes 


about. Consciousness is what makes us who 
we are. We wake up in the morning with 
a sense of self and experience external and 
internal sensations. Awareness is the essen- 
tial feature of consciousness, and there is 
some level of consciousness even in our 
dream states. So consciousness is related to 
our global or overall mental situation, where 
the storm of sensory stimuli have been suit- 
ably arranged, filtered, and averaged. 

We know from neurological studies of 
brain-damaged individuals and from func- 
tional imaging (MRI and positron emission) 
studies that the brain is organized in spatially 
defined modules. These modules respond to 
different aspects of the visual field that are 
mapped onto various venues of the brain. 
For example, the recognition of properly ori- 
ented faces is localized in the fusiform gyrus 
face area, and damage to this area results 
in prosopagnosia, the inability to recognize 
faces. The problem we are faced with is that 
the brain is subjected to a vast number of 
stimuli each second, and these stimuli are 
somehow processed in various locations; 
yet the brain has learned how to select the 
essential elements and to present an under- 
standable illusion of a three-dimensional 
world with colors, sounds, odors, tastes, 
and tactile sensations all appearing sta- 
ble and natural to us. Furthermore, we 
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can move and respond in “natural” ways 
in this self-centered awareness. So where 
does this blending and integration come 
from? No areas of the brain have been 
identified with the construction of con- 
sciousness, even though consciousness is 
modulated by many parts of the cerebral 
cortex. 

Where does art fit in? One view is that 
art is an extension of what the conscious 
mind is doing for us. The mind selects 
essential features out of cacophony and 
chaos that represent a consistent view of 
the world, though not necessarily objective 
reality. We believe the mind “enjoys” under- 
standable presentations and is disturbed by 
unresolved sensations. Things tailored to 
the capabilities of the brain attract rather 
than repel human beings. Similarly, effec- 
tive art represents the product of millennia 
of experiments by artists to discover pat- 
terns and designs that stimulate the cerebral 
cortex. Rather than studying the structure 
of the brain in an attempt to discover the 
origin of art, artists and other experimen- 
tal neurologists discover art by trial and 
error. The relevant structures in the brain 
are revealed by those compositions that are 
judged to be art. Feelings of joy and even 
euphoria are felt by the mind when “beau- 
tiful” human creations are experienced, 
whether sculpture, paintings, music, or 
theories of the universe. 


18.3 How Images Are Perceived 


he problem here is to understand 
how images received by our visual 
system are processed to create the 
world we perceive. There are two stages of 
perception that I will consider separately. 
The first has to do with automatic, fast vision 
capture and processing. These are processes 
we share with other primates. Our eyes do 


not make very good images. They only have 
reasonable resolution in the center of the 
visual field, and this part must be projected 
onto the only area of the retina that has good 
resolving power, namely the fovea centralis. 
Our vision relies on a coordinated system 
of extraocular muscles to orient our eyes 
and direct our focus to points of interest. In 
addition to these limitations, the amount of 
brain power that can be devoted to vision 
is not unlimited. It has long been realized 
that the 2D images projected on our retinas 
do not alone contain enough information 
for the creation of the 3D world we per- 
ceive. The task of uniquely determining the 
features of an object from its image on the 
retina is an example of the inverse problem, 
which turns out to be indeterminate. Our 
visual system copes with these limitations 
by judicious choice of a scanning path for 
our eyes over the visual field coupled with 
perception aided by unconscious inductive 
inference, that is to say by statistical knowl- 
edge of prior experiences. Here scanning 
path refers to the sequence of focus points 
chosen to assess a scene. This overall pro- 
cess is known as bottom-up-awareness.”*° 
Think of built-in defaults: just as our 
computers can be programmed to recog- 
nize a few letters and jump ahead to a likely 
word, the brain interprets images with a bias 
toward likely outcomes. For example, at first 
glance all objects are assumed to be viewed 
from the top and illuminated from above, 
and faces are quickly recognized if they have 
the proper orientation. Similarly, missing 
pieces are supplied to figures, ambiguous 
letters are interpreted according to context, 
and objects are grouped by proximity and 
similarity. We also get an immediate impres- 
sion of depth. At close range this is based 
primarily on the slightly different images 
received by the two eyes (binocularity). 
There are also kinetic clues to distance as we 
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FIGURE 18.2. A photograph of a pill case with two orientations. 


move relative to objects in the visual field. 
Nearby objects appear to move faster than 
distant objects in a way that enhances our 
sense of depth (motion parallax). A major 
part of this effect results from the chang- 
ing occlusion of distant objects. Figure 18.2 
illustrates our built-in assumption about 
lighting with a photograph of a package of 
pills from the pharmacy. The right and left 
sides are the same photograph with differ- 
ent orientations, and most people interpret 
the right side as bumps and the left side as 
pits. 

For distant scenes and 2D representa- 
tions, there are numerous (monocular) 
clues to depth. Our default is linear perspec- 
tive, where similar-sized objects appear to 
decrease in size with increasing distance 
but still are perceived to maintain their size. 
This can lead to errors and illusions. Also, 


our natural orientation on the surface of the 
earth leads to the assumption that elevation 
is related to distance, and we note that chil- 
dren tend to place distant objects high in 
their drawings. Of course, even in 2D, dis- 
tant objects are occluded by near objects, 
and atmospheric effects on clarity and 
contrast often distinguish distant objects. 
Other subconscious clues come from shad- 
ows and even the orientations of shapes and 
forms. A few visual illusions based on clues 
for distance are shown in Figure 18.3.” 
Still other illusions are not so easy to 
explain. In Figure 18.4, I show the 19th- 
century Múller-Lyer and Poggendorff 
illusions in parts (a), and (b), respectively, 
and a vase-or-faces construction in part 
(c). In part (a), the two lines are of equal 
length, and the illusion of unequal lengths 
persists even when the arrow heads are 
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FIGURE 18.3. Visual illusions: (a) converging straight lines appear to be parallel, (b) two hori- 
zontal lines (yellow) have the same length, and (c) the red and black oblique lines are parallel. 
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FIGURE 18.4. More visual illusions: (a) Muller-Lyer, (b) Poggendorff, and (c) vase or faces. 


replaced with circles or squares. In part (b), 
the oblique lines exactly match up regard- 
less of how it appears. According to recent 
studies, both illusions can be explained on 
the basis of statistics of what the visual sys- 
tem has encountered in previous scenes. 
Reactions to part (c) apparently depend on 
what the observer is prepared to see; except, 
of course, in cases where the viewer is famil- 
iar with the type of illusion. Readers can 
find numerous still and animated illusions 
online at Wikipedia and other sites.!”** 

Inference drives many of our percep- 
tions and leads us to see or imagine familiar 
objects in the shapes we see. Contours that 
appear to represent one thing but then shift 
to a different appearance when a different 
side is studied provide examples. Another 
set of illusions result from the fact that our 
visual “where pathway” (Chapter 15) makes 
use of illuminance, or perceived brightness, 
rather than color in establishing position. 
For example, objects that have a different 
color than the background but the same 
luminance may appear to have an unstable 
position. 

In bottom-up processing, salient fea- 
tures of a scene receive immediate, focused 
attention or fixation, and the scan path of 
our vision quickly samples the image and 
returns to more interesting features for mul- 
tiple hits. Studies of eye motion reveal that 
faces are high on the list of salient objects. 
Saccadic eye movements, which consist of 
angular jumps up to about 60° each requiring 


20 to 200 milliseconds, direct both eyes toa 
sequence of visual points. This generates a 
few high-resolution snapshots from foveal 
vision per second while the data from fuzzy 
peripheral vision is simultaneously received 
at much higher data rates. It is believed that 
this unbiased behavior of the visual system 
provides more or less the same impression 
of a scene to all observers. 

The second stage of visual perception 
involves intentional direction of vision, i.e. 
volition. Here the “searchlight” of attention 
is brought to bear on features according to 
personal schemata. This is known as top- 
down awareness. Schemata are essentially 
schemes for organizing knowledge based 
on what is important to an individual. 
Training, background, and even role play- 
ing can affect the sampling and analysis of 
an image. This implies a “mind set” that can 
lead to increased efficiency but also auto- 
matic, “knee-jerk” assessments. At its worst, 
this can result in rigidity and reluctance 
to accept change, the opposite of open- 
mindedness. For example, we see schemata 
in fads that dictate which styles are “in” dur- 
ing a given historical period. Schemata are 
not only locked in by experience or incli- 
nation, they may exist temporarily because 
of a recent work experience or even a role 
playing exercise. For example, you might be 
asked to assume that you are a policeman 
while viewing a scene. 

In viewing a work of art, the initial visual 
scan (bottom-up awareness) may emphasize 
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faces and other salient features to provide 
an initial impression. The immediate impact 
of the work may result from bright colors or 
perhaps novel features. In the second stage 
(top-down awareness), the observer can 
bring to bear education, experience, and 
training to evaluate the work in context. At 
this point discussion with others and addi- 
tional observation may grossly affect the 
assessment of the observer, and personal 
schemata may be modified or strengthened. 
Art judges, like all of us, have sets of sche- 
mata in place. In a given period, most of the 
judges may have similar schemata, and that 
poses a problem for new styles. There is also 
the schema of the jaded judge who is tired of 
images of sunrises, sunsets, hot air balloons, 
blurred water, and so on. 


18.4 Why Do Images 
Attract or Repel? 


ow we come to the crux of the 

matter. Why do we respond as 

we do to art? It is easy and fun to 
speculate. Consider the following suggested 
“attractants”: 


Representations and reminders of stimu- 
lating experiences: Representational art 
certainly reminds us of people and events. 
Food, games, combat, and attractive people 
are all featured in popular art. This brings to 
mind good documentary photography that 
has impact and evokes feelings. At a deeper 
level all art represents nature, and some 
universally satisfying features of images 
reveal our evolutionary past. Natural 
landscapes are preferred, especially ances- 
tral landscapes like the African savanna. 
Furthermore, images of natural objects such 
as mountains, clouds, and trees often reveal 
patterns that repeat at increasing magnifi- 
cation. For example, shorelines may have 


a similar appearance, at least statistically, 
when viewed from 1000 ft., 5000 ft., and so 
on. The complexity of these images can be 
characterized by a similar “fractal” dimen- 
sion, and it turns out that observers prefer 
images that have the fractal dimension of 
nature even for things like city skylines. 


Sexual stimulation, overt or subconscious: 
At the simplest level images may remind 
the viewer of sex objects or sexual activi- 
ties that are stimulating. The suggestions 
may be more subtle, as in, for example, hid- 
den faces or subliminal images in artworks. 
It has been suggested that Stone Age cave 
paintings contain abstract representations 
of vulvas. At a different level, art may serve 
as the plumage of the artist to impress 
potential mates, a la the ostrich. Similarly, 
the ability to appreciate or even afford art 
can confer status. 


Curiosity and inquisitive tendencies: 
Animals tend to explore neighboring areas 
even when food is plentiful in the vicinity. 
This can have survival value when environ- 
mental conditions change, and curiosity 
appears to be an essential feature of our 
genetic makeup. This may explain the 
stimulating effect of art, since art involves 
novelty as well as repetition. In our search 
for novelty we travel to new places, visit 
museums, and sometimes even attempt to 
create art. This line of thought fits well with 
the idea that all forms of fictional experience 
including art, drama, and imagination play 
an important role in organizing the brain— 
a major adaptive task throughout life.’ 

This list is intuitive and may capture 
important attributes of art, but it is in the 
category of suggestions rather than proofs. 
Experience shows that the brain is not only 
capable of creating art and judging art, but 
that it is inclined to commit its resources 
in those directions. Since the late 20th 
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century functional imaging of the brain by 
neuroscientists has revealed details of the 
operation of the visual system and has pin- 
pointed areas of the brain that are involved 
in various tasks. This is very encouraging to 
those who are attempting to discover the 
biological basis of art creation and apprecia- 
tion, but we are very far from being able to 
understand anything about our emotional 
response to art from physical studies of the 
mind in action. 

At this stage we must be satisfied with 
a descriptive approach based on catalog- 
ing the way the mind perceives and reacts 
to art. This is an experimental study, 
and through the millennia artists have 
unknowingly served as neuroscientists in 
determining which constructions and com- 
positions stimulate the mind. Our museums 
are filled with works of art that have stood 
the test of time, and every day new works 
of art including all varieties of photographs 
are competing for approval. The take-home 
message is that we already have a vast col- 
lection of works of art that have been found 
to stimulate the mind, and this reveals to us 
the way the mind functions. We can use the 
successful features of works of art in new 
creations, and we are free to experiment 
with new art forms to determine their com- 
patibility with the structure of the human 
mind. Humans have very similar visual 
capabilities, but our schemata are all dif- 
ferent. This accounts for subjectivity in the 
evaluation of art, but, in spite of that, there 
is wide consensus on what represents good 
art. 


What we can learn from successful images: 
Just as the conscious mind thrives on the 
consistency and stability of our world view 
in spite of the plethora of sensations and the 
jumpy nature of eye movements, it responds 
favorably to simplicity in framed images. 


That does not mean that details should be 
missing, but rather that there should be a 
center of attention and a minimum of dis- 
tractions. The eye should be directed to the 
most important areas by visual clues and 
should not be confused by multiple areas of 
equal importance. Similarly, framed images 
that are balanced and are not boring are 
favored. This has to do with composition, 
that is to say the location of features that 
have been selected for inclusion by the artist 
and their effect on the overall impact of the 
work. Recognition of these features has led 
to rules of composition. 

Beyond the simple admonitions to keep 
it simple and avoid distractions, we can list 
features that have found favor with most 
observers. These guidelines to composition 
can save time and, to some extent, substitute 
for experience for beginning photographers. 
With digital photography, where each cap- 
tured image costs practically nothing, it is 
all too easy to shoot away without planning 
and later to realize that the compositions 
lack something important. I believe that 
lists of guidelines can increase awareness 
about composition and motivate planning 
by photographers. They may also forewarn 
about likely reactions of critics. 

With the usual caveat that guidelines are 
not rules and that there are exceptions to all 
of them, I list a selection of guidelines that 
are frequently encountered. 


1. Establish a major object or area of 
interest. It should be relatively easy to 
answer the question, “What is the sub- 
ject of this photograph?” The object of 
interest can be isolated by means of 
placement, background, and depth-of- 
field. In some cases, natural lines and 
contours can be used to direct the eye 
to the subject. Natural frames within 
the image can also be helpful. 
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2. Avoid distractions. Some of the major 
distractions are competing points of 
interest such as bright areas (espe- 
cially at the edges), large fuzzy areas 
in the foreground, busy backgrounds, 
and perhaps lines that run directly to 
a corner. 


3. In general, avoid placing objects 
directly at the center of the frame 
unless size and symmetry demand it. 
Center placement, especially of small 
objects, makes balance difficult and 
tends to be boring. As a corollary, 
avoid placing the horizon exactly in 
the center of a picture. The subtle dif- 
ference in composition between Figure 
18.5(a) and Figure 18.5(b) is sufficient 
for most observers to find (b) more 
pleasing. 

4. Consider using the “rule of thirds.” 
Divide the image into thirds in both 
the horizontal and vertical directions 
by means of imaginary lines to make 
nine blocks. The crossing points of 
these lines, namely one third of the way 
in from both the vertical and horizon- 
tal edges, make favorable “hot spots” 
for the placement of objects or points 
of interest in the image. Placement at 
a crossing point works for the geyser 
in Figure 18.6; however, the symmetry 
and size of the orchid in Figure 18.7 
leaves little choice about placement. 


5. Give moving objects extra space in 
front for their anticipated movement. 
This is obviously desirable in most 
cases. In the event that the moving 
object has a “tail such as the wake for 
a boat or the jet trail of an airplane, it 
may be better to assume that the tail is 
part of the object. 


6. Avoid awkward clipping of objects 
or features at the edge of images. For 
example, windows in walls should usu- 
ally be totally in or out of the frame. 
Tight cropping to show a face or per- 
haps a waist-length portrait may be 
fine, but avoid clipping small parts. For 
example, try not to clip a hand or an 
ankle and foot. 


This list simply attempts to put into 
words some of the features that contrib- 
ute to our like or dislike of certain images. 
Sometimes placement is so obvious to most 
people that a rule is superfluous. Consider, 
for example, classic portraits created over 
many decades of faces that show two eyes. 
The faces are seldom straight on, and the 
dominant eye tends to be in the exact center 
of the canvas with remarkable consistency. 
The location of noses and mouths is much 
more variable. This observation is interest- 
ing but not particularly helpful. 

With all this discussion of composition, 
one should not lose sight of the importance 
of subject matter. A valid criticism of any 
work of art is that there is nothing inherently 
interesting in the subject. This immediately 
gets us into the most subjective part of art 
criticism. One photographer may find art in 
the placement of a cigarette butt on a side- 
walk or the location of a weed growing from 
a crack in concrete, while another photog- 
rapher is bored stiff by the same subjects. 
The choice of the subject in a photograph is 
quite personal. I personally am attracted to 
images that help me see the world in a new 
way. That might mean detail and colors in 
insects and birds, frozen action in nature, 
or composition and color in landscapes. The 
reader can insert his or her choices here as 
well. 
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FIGURE 18.6. Castle Geyser and the rule of thirds. 


FIGURE 18.7. Orchid symmetry controls the composition. 
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18.5 How Knowledge of the 
Visual System Can Enhance 
the Artist’s Bag of Tricks 


ccording to the aphorism, “knowl- 

edge is power,’ and knowledge of 

the characteristics of the human 
visual system should impart some advan- 
tage to photographers and other artists. 
I suggest two strategies for the use of this 
knowledge. The first and more important 
one has been used by successful painters 
perhaps without their understanding. Thus, 
artists have given conventional images extra 
impact through the use of unrecognized illu- 
sions. For example, impact can result from 
the use of equiluminant objects and back- 
grounds.! A commonly cited example is 
Monet's Impression, Sunrise, which is shown 
in Figure 18.8. When properly reproduced, 
the solar disk and its background have the 
same luminance, and for many observers 


the sun attracts attention because our visual 
system has a hard time fixing its position. 
There are also illusions of motion result- 
ing strictly from the geometry of an image, 
and in particular from closely spaced lines 
of high contrast. High contrast layers adja- 
cent to equiluminant layers can impart the 
impression of motion. Here again Monet 
comes to mind for his genius at making 
water appear to flow in his paintings. I have 
put together an illustration of this effect in 
Figure 18.9, where I hope the reader will 
experience a sense of flow in the solid areas. 
Still in the category of conventional 
images we find images hidden images within 
pictures that take advantage of the human 
propensity to recognize faces from a minimal 
set of clues.?* A famous example is Salvador 
Dali’s The Slave Market with Disappearing 
Bust of Voltaire (1940). Numerous examples 
can be found online, but as far as I know they 
have been limited to paintings. However, 


FIGURE 18.8. Monet's Impression, Sunrise. 
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FIGURE 18.9. Impression of flow, a visual illusion. 


with the capabilities of readily available 
photo processing software, the introduction 
of hidden images is within easy reach. 

The second category of images, sug- 
gested by the operation of the human visual 
system, are images that have been designed 
to have shock value by defying the built-in 
assumptions of our bottom-up awareness 


system (inductive inference engine). By 
that, I mean images that at least at first 
glance appear to represent impossible or 
extremely unlikely situations. Images that 
provide contradictory clues for determin- 
ing depth are easy examples. In Figure 
18.10, three identical images of an egret are 
located in a beach picture to give conflicting 


FIGURE 18.10. Identical images of an egret in three locations. 
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FIGURE 18.11. Moon space, a hole in reality. 


clues about size. The basic rule is that the 
perceived size of an object projected on the 
retina is proportional to the perceived dis- 
tance to that object. 

Visual dissonance is a state of psycho- 
logical tension that results from seeing 
something that is different from what one 
expects to see. Functional imaging studies 
show that this state leads to the activation 


of more areas of the brain than would be 
required for the processing of expected 
scenes. This increased stimulation may lead 
to attraction or repulsion, but at least it is 
stimulating. Surreal art such as that created 
by René Magritte and Man Ray provides 
numerous examples.” In Figure 18.11, I 
show a composite photograph inspired by 
the work of Magritte. 


FIGURE 18.12. The author with his camera. 
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Here the illusion of the moon over the 
Sandia Peak is disturbed by a look at the 
dark space. Another example of disso- 
nance results from impossible reflections 
in mirrors. A fanciful mirror photo of the 
impossible is shown in Figure 18.12. 


18.6 Reflections on Art 
in Photography 


why composition matters in photog- 

raphy and other visual art. Evidence 
indicates that art is closely related to and 
maybe inseparable from consciousness. 
The organization of the human mind to 
permit full color, binocular vision, and 
probably awareness as well, taking advan- 
tage of the full capabilities of our senses, 
requires years to develop. Our imagination, 
including all forms of art, certainly aids in 
the organization process. The result of this 
process is the mature mind that experiences 
pleasure and even exhilaration at suitably 
arranged representations of the world. We 
describe such arrangements as art and are 
especially delighted when such representa- 
tions expand our understanding through 
thought- provoking novelty. 

Our enjoyment of our colorful three- 
dimensional world is tempered by the knowl- 
edge that our conscious world is an illusion. 
This illusion replaces the flood of stimuli 
from our senses with a calm, stable world 
view. Furthermore, our conscious state 
attempts to minimize our visual defects and 
fill in unknown parts, such as blind spots, by 
sophisticated guesswork. This is, of course, 
the source of so-called visual illusions. As 
observers of art we are limited, both by the 
universal bottom-up-awareness and by our 


[+ tried to confront the question of 


own unique top-down awareness. 
Art exists because the human mind 
enjoys stimulation by visual images, and 


the response of our visual system is suf- 
ficiently universal that we can share the 
pleasures of visual aesthetic stimulation 
with others. Artists through the millennia, 
and neurologists more recently, have dis- 
covered empirically what types of images 
appeal to the human mind, and the experi- 
ments continue to our great benefit. What 
I have learned from this study is something 
about how the human mind copes with the 
limitations of our visual system, and more 
importantly that there is a vast world of 
wonderful art to be enjoyed. And last, but 
not least, there is still a world of wonderful 
art yet to be created by us photographers 
and other artists. 
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CHAPTER 19 


What We See and 
How We Photograph It 


Reality is merely an illusion, 
albeit a very persistent one. 
—ALBERT EINSTEIN 


here is no reality—get over it. 

I don’t mean that the real world 

doesn’t exist with all its objects, 
scattered and absorbed radiation, etc., but 
rather that we only “see” a very small part of 
what is in front of us, and that part is heavily 
filtered, selectively enhanced, interpreted, 
and corrected. In short, we experience a col- 
orized illusion or, better, a simulation of the 
world that is “visible” from our perspective. 
And that simulation is about 100 millisec- 
onds out of date by the time we consciously 
experience it, except for selective parts that 
have been extrapolated in time by the mind 
to predict the “now” This and every other 
comparison of “reality” with perception is 
controversial.! However, useful predictions 
or “future seeing” based on experience 
should come as no surprise. Evolution works 
to increase our survivability rather than to 
record accurately all aspects of our environ- 
ment. Our memories are designed the same 
way. At best they provide instructions for 


getting along in the world. “Our repository 
of memories is less like a library and more 
like Wikipedia, where each entry is open to 
editing anytime it’s pulled up”? 

In the following sections I briefly review 
the operation of our visual system and 
approach the question of where that leaves 
photography. In other words, what is left 
for photography and what can photography 
achieve? I believe there are two paths, which 
are not exclusive. First, there is the quest for 
the ultimate re-creation of reality. The end 
result would be a spherical, 3D, panorama 
of the world that would be instantaneously 
corrected and updated for our field of view 
and point of focus. This imitation of our 
real-world stimuli could perhaps be experi- 
enced with the help of virtual reality goggles 
or their future counterparts. In this path 
there would be nothing left for the artist or 
the hobbyist. I call the other path the artis- 
tic pathway. In the artistic path there would 
be self-imposed restraints as in traditional 
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art. The artist does not attempt to capture 
the entire world on one canvas, and sculp- 
tors limit themselves to one or at most a few 
figures. Before I continue along this line of 
thought we need a few details. 


Vision The eyes and brain permit the simu- 
lation of the real world by the mind. The 
simulation is not designed by evolution to 
show realism, but rather to enhance sur- 
vivability. The primary visual cortex is not 
a correlate of consciousness. The response 
of cells in our retinas and our visual cor- 
tex have some correlation with our visual 
experience some of the time, but not always. 
In other words, we do not see with our 
eyes! Christof Koch, in his insightful book 
Consciousness has described three ways we 
know this to be true.’ 

First the image on the primary visual 
cortex is jumping every second or so as our 
eyes dart from point to point (saccades) 
(Figure 19.1). At this stage, vision would 
resemble the output of a video camera that 
is quickly changing direction. Second, each 
of our eyes has a significant blind spot in the 
place where the optic nerve leaves the ret- 
ina. We are normally not conscious of this 
omission, because the brain fills in the miss- 
ing image by experienced guesswork. And, 
finally, we see colored images with great 
reality in some dreams even though our eyes 
are closed. Processing of the stimuli from 
the visual cortex occurs in three dozen dif- 
ferent regions of the brain by modules that 


detect edges, vertical, horizontal, and on a 
slant. Motion is detected, and so are spatial 
frequencies. In addition, color is assigned 
to patches in each image to insure that we 
distinguish things necessary for survival in a 
consistent way. The colors assigned to each 
patch, e.g. a leaf, exhibit constancy even 
when the light scattered from that patch has 
a different wavelength. Luminosity is also 
manipulated in unexpected ways (Figure 
19.2). 

This processing has built-in priorities 
and capabilities, such as the learned abil- 
ity to recognize human faces in the proper 
orientation and for interpreting scenes with 
the assumption that light incident from 
above. It is estimated that all of this pro- 
cessing requires about 100 ms before visual 
information can be sent to the prefrontal 
cortex and thus appear in our conscious 
view of the world (Figure 19.3). 

Visual information reaches the uncon- 
scious mind somewhat earlier, and there is 
editing in the short-term memory and the 
long-term memory. In addition to this direct 
picture, we must realize that everything is 
not strictly linear in time. Apparently, the 
mind corrects when possible for process- 
ing delays by extrapolation so that we can 
react to instantaneous necessities such 
as dodging a thrown rock (Figure 19.4). 
Occasionally, the correction is wrong, and 
the memory must be rewritten. All of this 
has been verified by experiment. Numerous 
examples of illusions in space and time can 


FIGURE 19.1. Eye movement when examining a figure with instructions (Yarbus, 1967).4 
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FIGURE 19.2. Contrary to appearance, the squares marked A and B are the same shade of gray. 


©1995, Edward H. Adelson. 
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FIGURE 19.3. Sketch of the human brain. The optic nerves, lateral geniculate nucleus, and con- 


nections to the visual cortex are not shown. 


be found online at illusionoftheyear.com. 
Limitations of vision, visual illusions, and 
timing corrections are commonly used by 
magicians to perform their “magic.” Some 
of this is described in the delightful book 
Slights of Mind on neuromagic by Macknik 
and Martinez-Conde.5 

So what does all this have to say about 
our reaction to, and our appreciation of, 
art? This brings us to recent work on the 


responses of the conscious and unconscious 
mind to works of art. The most complete 
and up-to-date account of the status of this 
work can be found in Eric Kandel’s book 
The Age of Insight. Art evokes emotions and 
stimulates the pleasure centers in the brain. 
It takes advantage of the brain’s ability to 
recognize objects and people from sketchy 
outlines, to respond to exaggerated features 
in drawings of faces and anatomy, and to 
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FIGURE 19.4. (a) The ball rolls to the left and the lamp flashes when the ball is directly below 
it. (b) An observer is aware of this flash 100 milliseconds later and reports that the ball is well 
to the left of the lamp when the flash occurs. We conclude that the mind projects the ball where 
it is expected to be when it is actually seen. This is an illusion. 


respond to colors. We enjoy the sights and 
sounds of nature as well as urban scenes 
and the appearances of some people with- 
out conscious thought. And, as discussed 
in Chapter 18, we enjoy some forms of art. 
Scenes on canvas and in photographs often 
give pleasure. Why art and other forms of 
imaginative fiction give pleasure is con- 
troversial. Kandel suggests that feelings of 
well-being may arise because art predicts 
biological rewards.* His recent book on 
reductionism in art and brain science con- 
siders the emotional impact of figurative 
art versus that of abstract art.’ Gazzaniga 
thinks that fiction and art help organize the 
brain, a demanding adaptive task in human 
development.* 


Photography If vision ultimately pro- 
vides a simulation of reality and not reality 
itself, what can we say about photography? 
Photography provides an imitation of the 
visual stimuli we experience, but in a very 
restricted way. Photographic images are 
limited by most or all of the following; 


1. Two-dimensional presentation. Most 
of our sense of depth-of-field in vision 
results from stereopsis associated with 
binocularity rather than aperture-related 


depth-of-field. Close one eye and notice 
the apparent increase in the depth-of- 
field. After all, the focal length of the 
eye is only about 20 mm or close to 
that in the wide-angle lenses on our 
cameras. 


2. A fixed point of focus. The focus is 


fine for objects in one plane and 
deteriorates as a function of distance 
for objects not in the focal plane. 
The depth-of-field can be limited by 
increasing the size of the aperture in 
order to emphasize some features in 
an image, but this is a poor substitute 
for stereopsis. 


. Framed or boxed images. In the real 


world our eyes can look in unlimited 
directions (front-facing). The frame 
limits our view and focuses our atten- 
tion for better or worse. 


4. A fixed dynamic range that is limited 


both by a constant aperture for all 
parts of an image and by fixed sensi- 
tivity setting (ISO) of the sensor. In 
contrast, our visual apertures (pupils) 
adjust quickly, and our visual pigments 
adapt over time to different levels of 
light. 
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5. An inaccurate representation of color 
resulting from simulations using three 
primary colors or the extreme case of 
black and white presentations. Recall 
that no set of three monochromatic 
colors can match all the colors expe- 
rienced by the mind in combination 
with our set of visual receptors (cones). 


6. A fixed presentation that does not 
respond to our changing point of atten- 
tion. In the real world our eyes dart 
to a point of interest and that point 
becomes perfectly focused. Also the 
aperture adjusts over a wide range to 
give proper brightness in the selected 
area. 


Can these limitations be addressed and 
even corrected by technology? In most 
cases, yes. I consider the corrections in turn: 


1. We can capture sets of images from 
different perspectives in order to simu- 
late 3D vision. This will not completely 
capture our visual experience because 
it does not require lens accommoda- 
tion and change in relative orientation 
of the eyes that occurs when objects 
move relative to our location. 


2. New plenoptic cameras, for example 
the Lytro, can capture the light field 
rather than a set of intensities in a 
plane. The light field contains infor- 
mation about the direction of the rays 
reaching each point on the image, and 
this information can be processed 
after the exposure to adjust the point 
of focus to any desired plane. 


3. Aspherical panorama can be recorded 
so that the frame can shift to encom- 
pass any direction of view. This virtual 
reality technique is well-known and 
simply requires the collection of over- 
lapping images that cover all directions 
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in space. A 3D version of the spherical 
panorama is also possible. 


4. A collection of images taken with dif- 
ferent exposures can encompass the 
full dynamic range of an image. The 
images can then be combined to obtain 
the correct exposure for any point. 


5. In the case of color photography, we 
are close to matching all perceived 
colors, at lease in principle, by the use 
of imaginary color primaries that define 
a color gamut covering most of our vis- 
ual space. The imaginaries are simply 
colors that can be represented in a com- 
puter but cannot be created in nature. 
A more serious problem is our present 
inability to display the complete or near- 
complete visual gamut on a screen or in 
a print. Of course, the dynamic range of 
nature will always exceed our displays. 


6. Instruments can determine accurately 
our point of focus so that, at least in 
principle, the focus and brightness can 
be instantaneously corrected by the 
methods described above. 


Suppose that our goal is to stimulate our 
retinas in a way that closely emulates the 
stimuli we would experience in nature. In 
the future all of the corrections described 
above can be implemented with the aid of 
improved sensors, more powerful com- 
puters, and improved displays. From my 
early 21st-century perspective, I envision 
state-of-the-art virtual reality goggles that 
respond to the orientation of the head and 
the eyes. The resulting world view could be 
close to that presented to the camera on a 
Mars rover, an observer sitting on the edge 
of the Grand Canyon, or a camera hang- 
ing over Dettifoss in Iceland. This would 
be impressive and probably enjoyable, but 
it would not be art, and there would not be 
much left for a hobbyist. 
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But I do not that believe that future vir- 
tual reality presentations will diminish art 
or artistic photography. It has been said 
that art is what artists do with limitations. 
I know there are many quotations about 
exceeding your limitations and admitting 
no limitations, but history and successful 
artists, musicians, and writers teach us a 
different lesson. For example: 


No matter how gifted a painter, his work 
is defined by his limitations. 
ALAN FELTUS 


If you have five elements available use 
only four. If you have four elements use 
three. 


PABLO PICASSO 


What is not constrained is not creative. 


PHILIP JOHNSON-LAIRD 


It is interesting to consider the reaction 
of artists when photography appeared on 
the scene and co-opted the presentation of 
“realism” in scenes. Artists had discovered 
perspective and had learned how to work 
with light mixing as well as pigment mix- 
ing. Then photography seemed to have it all 
for the future. Rather than being depressed, 
Henri Matisse said, “The invention of pho- 
tography released painting from the need to 
copy nature” He saw this as an invitation 
to appeal directly to the emotions. Gustav 
Klimt responded in 1907 by throwing out 
conventional perspective and painting the 
glorious portrait of Adele Bloch-Bauer with 
beautiful decoration and a flattened space 
(Figure 19.5). This painting was stolen by 
the Nazis before World War II and was only 
returned to a Maria Altmann, a relative of 
the rightful owners, in 2004. The remarkable 
story of the recovery of the painting from 
the Austrian government is dramatized in 


FIGURE 19.5. Gustav Klimt’s portrait of Adele Bloch-Bauer with enlarged detail on the right-hand side. 
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the 2015 movie The Woman in Gold. In 
2006 this painting sold for $135 million. 
It can be seen in the Neue Galerie in New 
York. 

In the near future, photographers can 
look forward to cameras with improved 
resolution, greater dynamic range, and 
better color rendition. There will continue 
to be advances in the feature sets, which 
already can include 3D capture, automatic 
panoramas, high dynamic range capture 
and processing, timelapse, wireless, GPS, 
and so on. So one can go in many directions 
with photography, but in order to create art 
that will last, one would be well advised 
to find a niche and to concentrate in mas- 
tering that limited area. That might mean 
working only with a fisheye lens, infrared 
conversion, or with long time exposures. So 
many ways to focus attention, coupled with 
unlimited possibilities for combination and 
manipulation of photographic images, erase 
any boundaries remaining between art and 
photography. 

Is there a take-home message in these 
ramblings? I think there are two. First, the 
serious photographic artist should not fear 
what automated cameras with extensive 
feature sets will do to photographic art; and, 
second, we should be open to any kind of 
photographic processing to produce images 
with impact. Images should be judged on 
their own as art, and the methods employed 
should be disclosed only at the photogra- 
pher’s discretion. This follows from our 
realization that we always “see” the world 
through heavily manipulated images based 
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on assigned colors, enhanced edges, inter- 
preted shapes, extensive guesswork, and 
learned associations. 
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APPENDIX A 


A Historical Note 
on Enlargers 


hotographic enlargers have existed 
pe the earliest days of photogra- 

phy. The first enlarging system was 
described by Sir John Herschel in 1839. 
Since transparent negatives were not availa- 
ble until about 1850, all of the early enlargers 
were essentially large copy cameras. The 
problem for both copy cameras and nega- 
tive projection systems was to find bright 
enough light sources so that exposures could 
be made in a reasonable amount of time. 
One solution was to use sunlight, and solar 
enlargers known as solar cameras were in 
use with some success throughout the 19th 
century. The exposures were typically so 
long that the solar cameras had to be con- 
tinually adjusted to track the motion of the 
sun, and of course cloudy days were a prob- 
lem. Also, a variety of artificial light sources 
were used, the most popular of which were 
limelight, electrical arc, and burning magne- 
sium. Acetylene lamps were even used, but 


a series of explosions dampened the enthu- 
siasm for that light source. 

Even though Louis Jules Duboscq dem- 
onstrated an enlarger with an electric light 
(carbon arc) at the Paris Photographic 
Society in 1861, it was not really practi- 
cal for individual photographers because 
of the great cost of batteries and later of 
steam-operated generators. Thomas Edison 
demonstrated his incandescent light bulb in 
1879, but the time was not right for general 
acceptance because electricity was not gen- 
erally available. It was not until about 1912 
that enlargers with electric bulbs started 
appearing on the market for amateurs. This 
seems reasonable when one realizes that, by 
1907, only 8% of homes in the United States 
had electricity. The expense and hazards 
of early enlarger light sources have been 
recounted in great detail by Eugene Ostroff 
in his profusely illustrated review article. 
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Note 
1. Ostroff, E. (1984). Photographic Enlarging: 
A History. Photographic Science and 
Engineering, 28, pp. 54-89. 
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What Is Behind the Rules of Optics? 


You will have to brace yourselves for this—not because 
it is difficult to understand, but because it is absolutely 
ridiculous: All we do is draw little arrows on a 
piece of paper—that’s all! 

—R. P. FEYNMAN 


Iways in science we look for unify- 
As concepts and general theories. 
Light is an electromagnetic wave, 
so it should be described by Maxwell's the- 
ory of electromagnetism. But now we can 
go beyond that description and predict light 
intensity and the interaction of photons with 
electrons by means of quantum electrody- 
namics (QED). QED is a model of the world 
that is totally unfamiliar to most of us, but 
it is easy to see how it works without using 
advanced mathematics. R. P. Feynman, one 
of the founders of QED, explained in detail 
the basic ideas and how to use them in his 
beautiful little book QED, The Strange Story 
of Light and Matter. This brief appendix 
is an introduction to Feynman’s ideas. It is 
thinly disguised graduate level physics. I 
hope it will motivate serious science stu- 
dents to explore this topic through further 
reading. 
What we want to know is how light gets 
from a source to a detector such as those 


in the sensor of a digital camera. We know 
it has to do with waves and their interfer- 
ence. Fortunately, it turns out that a light 
wave can be represented by a rotating arrow 
(vector)—think of the hands of a clock. 
The important things to keep up with are 
the length (amplitude) and turn (angle) of 
the arrow. Each way that a photon can go 
from place to place is described by a sin- 
gle rotating arrow. Feynman proposed a set 
of rules for dealing with these arrows that 
permit us to compute the outcome of any 
optics experiment. He tells us not to worry 
about understanding them, just learn to use 
them, because they describe the way nature 
behaves. So here they are: 


1. If an event can happen in alternate 
ways, determine the arrows for each 
way and then add them by placing them 
head to tail. The “sum” is obtained by 
drawing an arrow from the tail of the 
first arrow to the head of the last arrow. 
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The length of this final arrow is called 
the “probability amplitude” 


2. The probability of the event is deter- 
mined by squaring the probability 
amplitude. 


What this says is that photons take 
every possible path, not just those speci- 
fied by Hero's law of reflection, Snell's law 
of refraction, etc. However, when the little 
vectors for all of the paths are added up and 
the sum is squared, there is little probabil- 
ity that a photon will violate the well-known 
laws of optics. This is known as Feynman’s 
sum-over-paths quantum theory, and it is 
consistent with the rules of optics and much 
more. The procedure, not why it works, can 
be understood by looking at some exam- 
ples. First, consider photons of red light 
going from a Source at S to a Detector at 
D as shown in Figure B.1. The wall between 
the source and the detector has two very 
small holes (slits) that are located at A and 
B. The holes are so small that the normal 
laws of optics do not work (diffraction 
effect). We start the clocks when the photon 
leaves the source and the clock hands (little 
arrows) rotate at the frequency of light until 
the photon reaches the detector. A photon 
moves at the speed of light, so the orienta- 
tion of the arrow will depend on the length 
of the path if nothing else happens along 
the way. Since the two paths have different 
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lengths, the travel times and the rotation 
angles will be different. 

Here is what happens. When the hole at 
B is blocked, the detector measures some 
light intensity, e.g. 100 photons per second. 
This is perhaps 1% of the photons that leave 
the source. The arrow representing this path 
has some direction, say 2 o'clock, and the 
length 0.1. With only a single arrow, the 
direction does not matter and the length 
squared gives 0.01 as the probability of 
detecting a photon. If B is open and A is 
blocked, the same intensity is detected as 
when only A was open. The arrow for the 
path through B has a direction, say 3 o'clock, 
and the length is again 0.1. Now for the 
good stuff. When both holes are open, the 
intensity can be anything between 0% and 
4%; and it depends on the separation of the 
holes as well as the frequency of the light. 
The exact number must be computed from 
the arrows. According to the rules, we con- 
nect the arrows and determine their sum. 
(See Figure B.2.) 

In this case, with arrows at 2 o’clock and 3 
o'clock, we find that the amplitude squared is 
0.198? or about 3.9%. This procedure can be 
repeated for different heights (vertical posi- 
tion) of the detector to map out the intensity 
pattern for the double-slit experiment. 

The reflection of light from a mirror, 
discussed in Chapter 6, provides another 
illustration of the sum-over-paths method. 


E 


Time 


PS 


FIGURE B.1. The two slit experiment. 
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Path SBD 
Path SAD (0.1) 
(0.1) 
Amplitude 
(Sum = 0.198) 


FIGURE B.2. The sum of arrows (vectors) for two paths. 


As shown in Figure B.3, light from the 
source can reach the detector by reflect- 
ing from any point on the mirror - though 
here we only consider a thin strip of mirror 
sticking out of the page. The length of each 
path is given by Equation (6.2), and the time 
required for the journey can be obtained by 
dividing the path length by the speed of light, 
c = 2.997 x 10%m/s. The result of this calcu- 
lation is shown at the bottom of Figure B.3 


for each reflection point. In this illustration 
the length of the mirror is L = 10 cmanda 
small section centered on x = 5 cm has been 
greatly magnified to show the clock angles 
for a region about 600 um long. When the 
small arrows are added, the length of their 
sum is found to be determined by a few 
arrows in the center. It would not change 
significantly if more arrows were added at 
the ends because they would only continue 
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FIGURE B.3. Reflection of light with wavelength 1 = 550 nm from a mirror. The length of 
the mirror is 10 cm and the source and detector are located above the left and right ends of the 


mirror, respectively. 
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to go in circles. Note that the spiral formed 
by the arrows is just the Cornu spiral found 
in optics textbooks. So the probability is 
nonzero for reflection from the vicinity 
of the point where the angle of incidence 
equals the angle of exit. This agrees with 
Hero’s law and Fermat’s principle of least 
time. It is, however, more general, since it 
correctly handles cases where the maxi- 
mum time path is chosen, e.g. reflection 
from an elliptical concave mirror. 

There is still more to recover from the 
sum-over-paths analysis of reflection. 
Suppose we chop off all of the mirror except 
the piece from x = 0 to x = 2 cm. We know 
that no light from that region is reflected 
to the detector, but it is still worthwhile to 
look at it in detail. If we take the angle of 
the arrow for reflection from x = 1 cm as 
the reference angle and examine the other 
angles for reflection in that vicinity, we find 
the result shown in Figure B.4. Every 0.4 
um (400 nm) the orientation changes by 
180°, and so the amplitude for reflection 
completely cancels out for that region, i.e. 
the arrows add up to zero. However, the 
diagram suggests how the reflection can 
be recovered. All we have to do is to scrape 
away the mirror at the positions where the 


arrows are aimed down! Actually, 3 o'clock 
to 9 oclock. When we add up the remaining 
arrows and square the sum, we find that the 
probability is large for reflection of 500 nm 
light to the detector. What we have done is 
to create a diffraction grating. When white 
light is used as the source, each wavelength 
(color) diffracts strongly at a different angle 
as shown by reflection from the CD in 
Figure 2.4. 

These examples show the origin of 
the rules of optics at a fundamental level 
including, when appropriate, Fermat’s prin- 
ciple of least time. However, there are still 
some questions about the interaction of 
light with matter that results in reflection, 
transmission, and refraction. The angle for 
each rotating vector depends on the total 
transit time through a lens or collection of 
lenses including the time inside glass where 
the refractive index differs from unity. 
Other things can happen along the way, as 
well, and the interaction of light with mat- 
ter (more precisely photons with electrons) 
requires a few more rules. To handle reflec- 
tion from a sheet of glass, when some of the 
incident light is transmitted, the arrow for 
the reflected light must be reduced in length 
so that its square equals the probability of 
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FIGURE B.4. Reflection from the end of the mirror far from the path of least time. 
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reflection. Also, the process of reflection 
(from air back to air) reverses the direction 
of the arrow, i.e. gives a phase shift of 180°. 
This reversal of direction also applies to the 
mirror we considered above, but in that case 
the reversal did not affect the result since it 
applies to every arrow. The phase reversal is 
essential, however, for dealing with sheets 
of glass where light can reflect from more 
than one surface, and the arrows for paths 
with different numbers of reflections have 
to be added. Another rule, apparent from 
previous comments, is that light appears to 
require a longer time to pass through a piece 
of glass than through the same distance in 
air, and the time in glass is approximately 
equal to the time in air multiplied by the 
refractive index. 

But now the number of rules is growing, 
and it is not clear where they are coming 
from. Actually, things are not as bad as 
they might seem. The phase shift on reflec- 
tion and the refractive index are, in fact, 
the results of calculations involving all the 
atoms in the glass. A single rule specifies 
the scattering of a photon by an atom. The 
arrow shrinks by an amount that depends 
on the kind of atom and also turns by 90°. 
I am not going to complete these calcula- 
tions, but simply tell you that when the 
arrows for the photons, scattered from all 
the atoms, in the direction of reflection, are 
added, the result is a wave at the surface of 


the glass that can be described by an arrow 
appropriately reduced in length and with a 
reversed direction. This same model of scat- 
tering of photons by atoms also produces a 
transmitted light wave that can be described 
by an arrow with exactly the same orienta- 
tion as would be calculated by assuming 
that the velocity of light inside the glass is 
reduced by the factor n. A detailed discus- 
sion of these calculations can be found in 
the Further Reading references. 

All of this stuff about rotating arrows is 
just a way to approximate the final quan- 
tum mechanical wavefunction. Of course, 
quantitative calculations require the sum- 
mation of very many closely spaced paths 
and, in fact, integral calculus must be used. 
The simplified diagrams and computations 
presented here are designed to reveal the 
basic ideas rather than to provide tools for 
exact computation. 
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APPENDIX C 


Derivation of the 
Lens Maker's Equation 


he lens maker equation for a thin 

| lens is easily derived from the equa- 

tions for the focusing powers of the 

two surfaces as described in Equation (7.1). 

The situation is illustrated in Figure C.1, 

where the radius of curvature for the second 
surface, R,, is a negative number. 

Now imagine that the two diagrams are 

overlapped so that the lines defining the 


central plane of the thin lens coincide and 
the region having refractive index n, is con- 
fined between the two curves. The result is 
that the image plane of the first surface lies 
on the right side of the second surface at 
the position q,. Therefore, the subject for 
the second optical surface can be defined 
as p, = —q,. The equations for the powers of 
the surfaces are then given by: 


Central plane 


Central plane 


FIGURE C.1. The two surfaces of a simple lens. 


240 oy APPENDIX C 


4,A, 7 A, 


R, Py Ey R, 


n; 43M, = n, A; 


(C.1) 


The equation for the thin lens is then 
obtained by adding these equations with n, 
= n. With p, = œ, we have q, = f, thus we 
obtain the lens maker's equation (Equation 


(7.3)): 
Dz) 


Note that this equation is accurate only 
in the paraxial limit where the light rays 
make small angles with the optical axis and 
the lens is thin. 

It is also interesting to consider the case 
where n, + n, This time we find that: 


(C.2) 


Setting p, = œ% establishes that the left- 
hand side of Equation (C.3) is equal to n,/ 
fp where f, is defined as the focal length 
in the region where the refractive index is 
n, Therefore, we obtain a new conjugate 
equation: 


(C.4) 


This equation, for example, applies to the 
human eye where the volume between the 
lens and the retina contains vitreous humor 
and n, = 1.337. By reversing the direction of 
the rays and setting q, = eo, it is easy to show 
that 1, /f, = n,/f,. 


APPENDIX D 


Gaussian Optics and 
the Principal Planes 


JE essence of Gauss's theory can be 
captured without complicated equa- 
tions. Using the ray in Figure 9.4(b) 
as an example, the argument goes as fol- 
lows. The ray on the left side of surface A, is 
described by its distance (x,) from the axis 
and the angle (a,) it makes with the axis. 
This description is denoted by the symbol 
(V): 


1 


The effect of surface A, on the ray is 
described by the “operator” R,. On the left 
side of surface A, the ray is described by V, 
and on the right side it is described by R, V, 
which we call V’. This is an important step, 
because I have introduced the concept of an 
“operator” An operator acts on the quantity 
to its right and produces a transformation. 
In short, I will just say that we multiply by 
R, from the left. This is a powerful idea 


because it can be extended to all the nec- 
essary steps of refraction and translation. 
Each refraction step has an operator R that 
depends on the power of the surface, and 
each translation step has an operator T that 
depends on the angle of the ray and the 
distance traversed. 

Continuing the example, the ray incident 
on surface A, is described by V, = TR,V, 
= TV, and the ray exiting A, is described 
by V; = R,T.,R,V,. All am doing is mul- 
tiplying from the left with the appropriate 
operator each time the ray passes through 
a surface or traverses a space. So far the 
operations only describe a single lens; and 
when D,, (in Figure 9.4) is small, it is a thin 
lens. At this point, all of the operations can 
be combined (multiplied) to obtain a single 
operator M,, = RoT oR, 

Now suppose that the compound lens 
contains n elements. The same procedure is 
followed with a new R for each surface and 
T for each translation and eventually leads 
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to the operator M . The operators M,, and 
M,, that describe the effect of the single thin 
lens and the compound lens, respectively, 
are not identical; but they can be put into 
the same form by introducing translations 
(spaces) at the beginning and the end of 
the sequence for the compound lens. In the 
same notation, that gives a new operator 


M, = T'M,,T, where M,,, has the proper 
form only when the translation operators 
are based on the proper principal planes, P 
and P. Some readers will recognize V as a 
2 x 1 matrix (column vector), and that the 
operators R and T can be represented by 2 
x 2 matrices. 
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A Macro Lens 
with Attachments 


he working distances and magnifi- 

cations were measured for a Sigma 

105 mm macro lens with various 
combinations of extension tubes, diopters, 
and a tele-extender. The results are plotted 
as magnification versus working distance 
in Figure E.1. For each combination, meas- 
urements were made at far focus (infinity) 
and closest focus. The results are indicated 
with dots connected by red dotted lines. 
The points on the connecting lines have no 
significance and only serve to connect the 
end points. Each line is labeled to show the 
presence or absence of extension tubes (12 
mm to 68 mm), diopters, and a Kenko 1.4x 
teleconverter. The red star shows the bare 


lens at close focus. Unconnected dots also 
refer to close focus. 

The results show that a teleconverter 
in combination with extension tubes gives 
high magnification and large working dis- 
tances. In each case, the extension tube was 
attached to the lens first. In other words the 
teleconverter was mounted directly on the 
camera body. The reversed 50 mm lens (20 
diopter) gives high magnification, but at 
the cost of working distance. In Table E.1 
I show the measured distances (see Figure 
12.3) and the associated magnifications for 
all the combinations of the macro lens and 
attachments. 
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Sigma 105mm Macro 
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FIGURE E.1. Characterization of a 105mm macro lens. 
TABLE E.1. 
Lens combinations and magnification 
105 mm Lens+ Object to Sensor (cm) Working Distance (cm) Magnification 
1 bare 31.4 14 1.01 
2 2-diopter-close 29.2 8.5 1.24 
3 12 mm ext-close 31.4 11 ME) 
4 20 mm ext-close 31.7 10.6 1.31 
5 36 mm ext-close 32.6 9.8 1.54 
6 36 mm ext-far 54.4 36.7 0.35 
7 68 mm ext-close 34.6 8.6 1.89 
8 68 mm ext-far 43.6 22.7 0.67 
9 1.4x Kenko-close 33.2 11.6 1.41 
10 12 mm+1.4x close 33.1 10.1 1.67 
11 20 mm+1.4x close 33.4 g7 1.83 
12 36 mm+1.4x close 34.4 SLi 2.16 
13 36 mm+1.4x far 56 35.8 0.49 
14 68 mm+1.4x close 8515 6.9 2.83 
15 68 mm+1.4x far 45.1 21.7 0.93 
16 50 mm rev close 26 1.6 225] 
117/ 50 mm rev far 23 4 199 
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Observing and Managing 
Color: Dealing with Color 
Vision Anomalies 


The Camera versus the Human Eye 


uperficially, the human eye appears 

to be similar to a camera. Here, looks 

certainly deceive because, other than a 
few shared components, the two differ both 
in operation and in purpose. Consider first 
the camera. A camera records a scene all 
at once and in an instant. What is actually 
recorded is a projection of the scene by a 
rectilinear or fisheye lens onto a flat sensor. 
Furthermore, the modern digital camera and 
the associated post processing computer 
algorithms are designed to: 


1. Provide maximum resolution with an 
adjustable depth of field. 


2. Provide enough brightness to reveal 
some detail in shadows while main- 
taining detail in bright areas. 


3. Match the color perception of the 
standard human observer by sampling 
the image at different wavelengths. 


When successful, this setup yields a 
simulation of standard human vision that 
stands up to inspection in any part on the 
image. The camera’s lens, sensor, and com- 
puter are optimized to match as much as 
possible the capabilities of the human eye/ 
brain system. The post-processing of cam- 
era images also attempts to replicate human 
acutance and color perception. This is, of 
course, all based on a standard observer. 

In contrast to the camera, our eye/brain 
system scans a scene in many jumps called 
saccades and processes the incoming stream 
of information to capture what is important 
for our well-being. About two-thirds of the 
human brain is devoted to processing visual 
information and its interactions with other 
senses. Dozens of visual field maps in the 
visual cortex are involved in the extrac- 
tion of information concerning motion in 
various directions, spatial frequencies, rec- 
ognition of faces, discrimination of light 
stimuli from random noise of the photo 
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receptors, separation of motion in the sur- 
roundings from eye and body motion, and 
so on.! The stimuli are processed for imme- 
diate action but also to present a coherent 
picture to the conscious mind. This involves 
assigning colors to objects, insuring stabil- 
ity of the presented scenes, enabling depth 
perception, and extrapolating motion so 
that the time delay in processing will not 
interfere with conscious actions. All of 
this amounts to the creation of an amazing 
simulation of the surrounding world that is 
based on limited information obtained by 
an imperfect visual system. 

Psychologists have emphasized that we 
see with our brain using input from the 
visual cortex, but that our visual images 
are so processed that they have little direct 
correlation with images from our eyes. In 
fact, the brain can produce perfectly real- 
istic scenes even when our eyes are closed. 
This is, of course, the origin of dreams and 
hallucinations. 


Do Our Eyes Even Permit All of Us 
to Perceive the Same Colors? 


he obvious question is “Do we all 
experience the same visual image 
when confronted with a scene?” If 
our eye/brain systems are essentially iden- 
tical, it is still an unanswerable question. 
When I say that an object is red, how do 
I know that you are experiencing the same 
color when you agree that it is red? An eas- 
ier question to answer is whether you and I 
can distinguish the same pairs of colors. For 
example, experiments have been designed 
to determine whether two observers will 
select the same amounts of red and green 
light in a mixture to match a given shade of 
yellow light. 
All photographic color matching systems 
incorporate the idea of a standard human 


observer. (See Section 15.3.) The standard 
observer is just the average of a number of 
observers who match colored lights in an 
experiment. The original standard observer 
was introduced in 1931 by the International 
Commission on Illumination (CIE). A group 
of about 20 individuals compared a screen 
illuminated with a pure color, i.e. a spec- 
tral color with a known wavelength, with a 
screen illuminated by the overlapping beams 
of red, blue, and green light from a selected 
RGB standard set. The observers adjusted 
the intensities of the three primaries until a 
match was achieved with the light from the 
test lamp. The 1931 experiment used a 2° 
field of view to just cover the human fovea, 
while the 1964 CIE supplementary standard 
expanded the field of view to 10 degrees to 
cover part of the periphery as well (Figure 
15.6). 

The CIE standard observer is a reasona- 
ble choice for designers and photographers, 
but unfortunately many people do not see 
exactly like the standard observer. Consider 
the structure of the human eye. All vision 
depends on our retina, the layer of tissue 
on the inside back surface of the eye. We 
have in common a set of about 125 million 
rod cells in our retina. These photo recep- 
tors have maximum sensitivity at about 498 
nm, and they are responsible for our night 
vision. An individual rod can detect a sin- 
gle photon of light, and our visual system is 
designed so that we can perceive a pulse of 
light on the eye with as few as seven pho- 
tons. However, at high light levels, the rod 
cells are totally saturated and make no con- 
tribution to our vision. This brings us to the 
light-receptive cone cells that are much less 
sensitive but can function in normal day- 
light. There are 6 to 7 million cones, and they 
are responsible for all human color vision. 
The cones are concentrated in the fovea, a 
disk less than 0.3 mm in diameter, arranged 
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so that they can also provide high-resolu- 
tion vision as well. It is in the cones that 
the differences in human color vision are 
found. 


What Is Color? 


olor is not a topic in physics. Pure, 

monochromatic, light is character- 

ized by its intensity and wavelength. 
The combination of pure light from two 
different sources simply gives a light mix- 
ture with two different wavelengths. To the 
standard observer, the combination of the 
two sources will be perceived as a single 
color because of the processing of stimuli 
from cone cells in the retina and the visual 
cortex. For example, a screen illuminated 
by yellow and blue lights will appear white 
or near white. Similarly, a red and green 
combination will be perceived as yellow. 
Color is assigned to an object by the brain 
on the basis of the wavelengths of both the 
object and its environment. But the full set 
of colors perceived by the standard observer 
requires that the retina contain three types 
of cones that are sensitive to different ranges 
of wavelengths. We say that the human eye 
contains red, green, and blue cones, which 
is just shorthand for long wavelength 
(L), medium wavelength (M), and short 
wavelength (S) receptors, respectively. 

The presence of red, green, and blue 
cones in human eyes and the structures 
of the cones themselves have now been 
established by many experiments, but their 
existence was not at all obvious when Isaac 
Newton demonstrated the separation of a 
light beam into its component wavelengths 
(colors) in 1664. The idea that there are dif- 
ferent kinds of photo receptors in the retina 
of the human eye was presented by the 
English polymath Thomas Young in 1801, 
and he hit the nail on the head by proposing 


that there are just three types without any 
experimental evidence to back up his idea. 
He knew that three colors of light seemed 
to be sufficient to produce the full array of 
colors perceived by humans, but he had no 
way to know that there were not four or 
more different types of receptors. 

In fact, mammals can have from zero to 
three types of cones.” Nocturnal animals 
have up to 100% rods to enhance night 
(scotopic) vision and thus experience mono- 
chromatic vision. Owl monkeys and marine 
mammals have only one type of cone and 
are called cone monochromats. New World 
male monkeys have two types of cones and 
60% of females have three types, while 
Old World monkeys and humans share 
the trichromic visual system. Reptiles and 
amphibians are usually trichromats as well, 
but birds and tropical fish have more com- 
plex color vision than humans. Apparently, 
mantis shrimp hold the record, with up to 
12 types of spectral receptors. Furthermore, 
in honey and bumblebees we find a visual 
range that extends into the ultraviolet. 

This excursion into the animal kingdom 
was intended to show that the perception of 
color must vary wildly, and it is just a con- 
struct of the eye-brain system. With humans 
we also find a range of different cone sets 
that include standard observers as well as a 
variety of types of color blindness and color 
anomaly. Anomaly can involve hybrid genes 
that produce subtle deviations from stand- 
ard trichromatic vision. In past centuries 
such deviations would have gone unnoticed, 
but now with color management meth- 
ods in photography we can detect subtle 
differences in color matching abilities. 

Now let's take a brief look at the human 
system of cones.’ The red, green, and blue 
visual pigments are all opsin molecules 
that contain a chromaphore, 11-cis-retinal, 
but differ slightly in their amino acid 
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FIGURE F.1. Sensitivity of human photo pigments.? 


sequences. Amino acid substitutions in the 
opsin chains tune the aborption spectra 
of the chromaphores and account for the 
wavelength shifts. The genes that express 
the red and green pigments are located on 
the X-chromosome, while the gene for the 
blue pigment is on chromosome 7. Figure 
F.1 shows the absorption spectra for the 
red (L), green (M), and blue (S) pigments as 
well as for the rods (dotted line). The pig- 
ments that are responsible for red and green 
absorptions have quite similar amino acid 
sequences and are said to be highly homolo- 
gous. This accounts for the similar spectra 
and the small wavelength shift between 
them. Furthermore, the genes for red and 
green are located side by side on the DNA 
of the X-chromosome with red coming first. 
The arrangement LM is normal, but usually 
there are two or more copies of the M gene, 
and this causes no problems. 


Genetics of Color Vision Anomaly 


he most common kinds of color 

vision defects occur when a red (L) 

or green (M) gene is completely 
missing.* A protanope is a person who 
is missing the L gene and a deuteranope is 
missing the M gene. In either case the indi- 
vidual cannot distinguish greens and reds. 
A simulation of these forms of dichromatic 
vision is presented in Figure F.2. Trianopes, 
who are quite rare, are missing the S gene. 
The loss of an L or M gene usually occurs 
by means of gene recombination that can 
occur during mitosis. 

For example, two DNA chains with the 
LM combination interact to produce one 
chain with only L and one with LMM, as 
illustrated in Figure F.3. A male receiving an 
X-chromosome with only an L gene is sure 
to bea deuteranope. Females, of course, have 
two X-chromosomes which are randomly 
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Trichromatic Vision 


Protanopia 


Deuteranopia 


FIGURE F.2. Simulated appearance of flowers for trichromats, protanopes, and deuteranopes. 


expressed, so that a missing gene on one 
X-chromosome can be made up for by the 
expression of the other X-chromosome. 
This is just the beginning of the possible 
types of red/green anomalous vision. The 
most common variations in trichromats 
result from the presence of hybrid genes. The 
hybrids result from gene recombinations 
that produce red genes with some green 
component (L/M), deuteranomaly, or green 
genes with some red component (M/L), 
protanomaly, as shown in Figure F.4. All of 
the red/green vision anomalies amount to 
8% to 10% of human X-chromosomes, and 
hence the same percentages of men with 
anomalous vision. Women, whose risks are 


L M 
= sew sl 
L 1 M — 


cut by having two X-chromosomes, have 
only about 0.5% chance of anomalous color 
vision. 

As a final example of visual diversity, I 
would like to consider a difference between 
“normal” trichromats that is revealed in 
color matching experiments. Two types of 
L genes that differ in a single nucleotide are 
found on X-chromosomes. This results in L 
photopigments that have either serine (ser) 
or alanine (ala) at position 180. The absorp- 
tion spectrum of pigments with ser are 
red-shifted by 4 nm to 5 nm. Males, having 
only one X-chromosome, have either the 
ser or the ala version of the L gene, while 
women can have either or both in the case 


FIGURE F.3. Unequal homologous gene recombination to yield on X-chromosome that is missing 


an M gene and one normal X-chromosome. 


L M 
Li m — 


FIGURE F.4. Unequal homologous recombination to yield X-chromosomes with hybrid genes. 
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of heterozygotes. In color matching experi- 
ments to determine the amounts of red and 
green light required to match a given stand- 
ard yellow light, some males require more 
red than others. This results in a bimodal 
distribution for males, while females exhibit 
either the matching selections shown by 
males or something in between for heterozy- 
gotes. Therefore, some women have four 
cones and exhibit tetrachromacy. Human 
tetrachromats potentially can distinguish 
colors better than trichromats. 


Dealing with Differences 
in Color Perception 


his discussion has shown that 

8-10% of human X-chromosomes 

encode color vision anomalies, and 
that there are even differences in color per- 
ception for “normal” trichromats. I find it 
strange that these differences are seldom 
mentioned in discussions of color manage- 
ment and color correction for photographs. 
Of course, photographers want to have 
their photographs critiqued and may want 
to sell them, so the opinions of “standard 
observers” are very important. Presumably, 
protanopes and deuteranopes are aware 
of their condition and have been dealing 
with it. But those with other more sub- 
tle anomalies may be unaware how much 
they deviate from “normal” trichromats in 
color perception. This is made more com- 
plicated by the metameric nature of color 
matching that is to say the same perceived 
color can be matched by combinations of 
different sets of primary colors. Even in 
the group of “normal” trichromats there 
are differences in color matching abilities 
that may result from different pigment 
densities in the retina or from age-related 
changes in eyes, e.g. cataract-related lens 
color. 


Most differences in the ability to dis- 
criminate colors are permanent, and we 
should take this into account when we pro- 
duce images that will be viewed by others. 
This applies both to those with deviations 
in color perception and those who create 
photographs and illustrations for a wide 
audience, including those with a wide range 
of color responses. Consider the following 
situations: 


1. Obvious red/green color blindness: 
This condition is found in up to 
about 5% of the population, and these 
dichromats must adapt to color stand- 
ards set by standard observers who are 
trichromats. In the post-processing 
of images, this does not have to be a 
major problem. Small enhancements 
of saturation, clarity, and vibrance can 
be made without reference to red/ 
green differences, and of course pho- 
tographs can be critiqued by friends. 


2. Subtle differences in the group of tri- 
chromats: Groups of individuals who 
behave differently in color matching 
experiments are said to be metameric 
observers. More differences may be 
found when light sources, the color 
primaries, are changed, and this 
brings us to the topic of modern dis- 
play technologies.? Some individuals 
may be happy with color presentation 
on CRT displays but not with high 
gamut LCD screens. In Figure F.5 the 
spectral distribution for a CRT type 
display is shown. The blue and green 
bands are very wide and the red bands 
are scattered over a wide range. These 
distributions can mask differences in 
the absorption maxima. In contrast 
to this, the RGB band widths may 
be quite small in LCD displays, as 
shown in Figure F.6. Accordingly, LED 
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FIGURE F.5. Illustration of CRT type emission spectrum. 
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FIGURE F.6. Illustration of LCD type emission spectrum. 
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monitors may accentuate distinguish- 
ing shifts in the absorption spectra of 
anomalous photo pigments. 


. Photographers, designers, and illustra- 
tors: This group needs to be sensitive 
to the ways their audiences will see the 
work they create. There are various 
aids to help avoid confusion and even 
danger for color-blind individuals. For 
example, the EIZO ColorEdge moni- 
tor simulates the way color schemes 
for websites, signs, maps, etc. will 
be viewed by those with red/green 
(protanopia and deuteranopia) and 
blue/yellow (tritanopia) color vision 
deficiencies. 
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APPENDIX G 


Capturing Photons with 
Photographic Film 


Te discovery of photography in the 
early 19th century was essentially 
the discovery that certain silver salts, 
e.g. silver chloride (AgCl), silver bromide 
(AgBr) and silver iodide (AgI), are affected 
by light and can be processed to amplify the 
effect and make it permanent. These silver 
salts, known as silver halides, form crystals 
that visibly darken when exposed to bright 
light; the process continues so that in the 
presence of light the crystals get darker 
and darker. This darkening effect is neither 
very sensitive nor stable and is not very 
useful for recording images. The enabling 
discovery for photography was that chemi- 
cal processing, after exposure, can convert 
some of the exposed crystals into grains of 
metallic silver and also stop the darkening 
process. Indeed, there was a mystery. Even 
a low level of exposure does something 
to the crystals so that they can be “devel- 
oped” later to obtain a silver grain record 
of the exposure. After exposure, the halide 


crystals contain a latent image which lasts 
until the crystals are developed. 

Today, after many decades of experimen- 
tation, technological advances, and (more 
recently) scientific understanding, we have 
black and white and color films that provide 
stable, high-resolution images. However, 
they are still based on latent images stored 
in silver halide crystals. Modern film con- 
sists of a sturdy base about 100 um to 200 
um thick, usually cellulose acetate, which 
serves to support multiple layers of emul- 
sion held together with gelatin. For example, 
Fujicolor Provia 100 color reversal film con- 
tains more than a dozen layers, starting with 
a protective layer on top and ending with 
an antihalation layer to prevent reflection 
of light back into the emulsion. The thick- 
ness of the emulsion ranges from about 10 
um to 20 um. 

Since the photon capture step in film 
is always the same, it is easier to begin the 
discussion with black and white film as 
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FIGURE G.1. Monochrome (black and white) film. The thickness is not to scale. 


illustrated in Figure G.1. The silver halide 
crystals are roughly one micron in diameter 
and are randomly distributed. 

In the commonly used silver bromide 
crystals, there is a lattice of silver ions (Ag*) 
and bromide ions (Br”), as shown in Figure 
G.2. An energetic photon can knock an elec- 
tron out of a Br” ion, and the electron has 
some probability of being captured by a lat- 
tice defect (shallow trap). Ifa mobile silver ion 
happens to be present in the same trap or to 
reach the trap while the electron is still there, 
it will be reduced to a silver atom. This is not 
sufficient to contribute to the latent image, 
because a single trapped atom is not sta- 
ble. However, the newly minted silver atom 
will continue to hop from trap to trap, and 
it may encounter one or more other atoms 


hv 


to form a stable microcluster. The impor- 
tant point is that a certain level of exposure 
produces a proportionate number of stable 
microclusters of silver atoms in different 
silver bromide crystals. 

The microclusters of silver atoms are 
called sensitivity centers or development 
centers, and they constitute the latent 
image. This latent image can be “developed 
out” by applying a developer solution with 
a reduction potential sufficient to reduce 
only those crystals containing microclusters 
to silver particles. In chemical terminology, 
reduction means adding an electron so that 
the silver ion (+1) is converted into metallic 
silver (0). There are usually two additional 
steps in film processing. First, there is an 
acid “stop” bath (usually acetic acid) to 


FIGURE G.2. Photo excitation of silver bromide crystal. 
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terminate development and then a “fixer” or 
“hypo” solution (usually sodium thiosulfate) 
to dissolve and wash away “unsensitized” 
crystals while leaving the silver particles 
(grains) in place. In this way a few silver 
atoms in a cluster can act as a trigger for 
the conversion of billions of silver ions to 
metallic silver. The silver particles remain 
in the emulsion and constitute a nega- 
tive image, since it is darker where there 
was more exposure. In going from latent 
image to silver image we effectively have a 
powerful photon multiplier. 

Silver halides are primarily sensitive to 
blue and UV radiation. Modern panchro- 
matic film is possible because of sensitizers 
that are added to the emulsion. The sensitiz- 
ers are dye molecules that are designed to 
absorb a certain color of light and to trans- 
fer electrons to a crystal grain. Sensitizers 
also make possible the construction of color 
film that can produce color negatives and 
positives (color reversal film). Color per- 
ception and representation are discussed in 
detail in Chapter 15. Here it suffices to say 
that the “colors” of light can be represented 
with combinations of red (R), green (G), 
and blue (B) light in an additive process. 
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The three colors that are necessary for use 
as pigments, or in any other subtractive pro- 
cess, are cyan, magenta, and yellow. This 
means that color film must start with the 
capture of red, blue, and green light and end 
up after development with cyan, magenta, 
and yellow dyes for positive or reversal film 
and the corresponding negative colors for 
print film. 

Figure G.3 presents a simplified pic- 
ture of the emulsion layers in a film such 
as Fujifilm Provia. Each of the light sensi- 
tive layers contains silver halide crystals 
along with appropriate sensitizers and 
dye-coupler molecules. All of the layers are 
sensitive to blue light, so the first layer is 
assigned to blue and is followed by a yel- 
low filter to absorb the remaining blue light. 
The second and third layers are assigned to 
green and red light, respectively, and con- 
tain the required sensitizers. When the film 
is exposed to light, the light produces latent 
images in the three layers with concentra- 
tions of sensitized sites that are proportional 
to the exposure by the three colors of light. 
Thus the light is a stimulus that is repre- 
sented in the film by latent images for R, G, 
and B colors. 


PROCESSED FILM 
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FIGURE G.3. Simplified cross-section of color reversal film. 
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The development process consists of the 
reduction of silver ions in sensitized crys- 
tals to metallic silver and the oxidization of 
the developer molecules, i.e. the removal 
of electrons. The oxidized developer mole- 
cules are then free to react with dye-coupler 
molecules to produce the required dyes in 
each layer. Films with this design are very 
practical now, but initially there was much 
difficulty keeping the dyes in their proper 
layers. For that reason Kodachrome, the 
first successful subtraction-process film, 
was quite different. In order to avoid the 
wandering dyes, Kodachrome was designed 
without any coupling-dyes in the film! It 
was essentially a multilayer black and white 
film with appropriate sensitizers, and the 
dyes were all added during processing. 
The result was an extremely complicated 
processing procedure with 28 steps and 
requiring large, specialized equipment 
and highly trained technicians. In addi- 
tion to providing outstanding color and 
fine grain, the Kodachrome system had 
two major advantages. First, all the cou- 
plers were removed during processing, 
thereby improving stability; and second the 
layers in the emulsion were much thinner 
than those required to hold dye couplers 
in competitive products, thus improving 
resolution. 

Other successful schemes for manipu- 
lating dyes in color images were based on 
bleaching dyes already in place. A good 
example is Cibachrome (now Ilfochrome), 
that was engineered by Ciga-Geigy Corp. 
for color prints. In this material, stable azo 
dyes are incorporated into the light sensitive 
layers. The development process selectively 
destroys dye molecules in the exposed 
areas. This results in vivid colors and archi- 
val prints. Cibachrome/Tlfochrome prints 
are among the best archival color prints that 
can be obtained. 


The evolution of color photography 
extended over about a hundred years. 
There was a flurry of activity in the early 
20th century motivated largely by the 
movie industry, and by the 1940s a num- 
ber of practical color films were available 
to photographers. In 1947 Polaroid Corp. 
introduced the instant camera with self- 
developing film, and in 1963 the color 
instant camera was announced. This was 
perhaps the most sophisticated applica- 
tion of organic chemistry at the time. It also 
demanded reliable and accurate mechanical 
designs to ensure smooth movement of the 
print and delivery of developer as soon as 
an exposure was made. This heroic achieve- 
ment served its purpose for decades, but 
now has been swept away by the digital rev- 
olution. The Polaroid Corporation ceased 
to exist (except in name) in 2001, and other 
film companies are regularly announcing 
cancellations of products and developing 
services. 


Historical note (Holography, music, and 
color photography): Early attempts to make 
color photographs were all based on the addi- 
tive color process in which black and white 
negatives of the same scene were obtained 
through red, blue, and green filters. Positive 
images were made from the negatives and 
then projected through the corresponding 
color filters to recreate the color scene. This 
method, first demonstrated by James Clerk 
Maxwell (1861), continued to be used with a 
variety of ingenious variations well into the 
20th century. The positive color method, 
at its best, was used by Sergey Prokudin- 
Gorsky to document the Russian Empire, 
1905-1915. A single plate version, in which 
the color filters took the form of colored 
grains of starch, was known as Autochrome 
Lumiere. It produced a type of color pointil- 
lism that is still popular with collectors. 
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Pure color without dyes: In 1908 Gabriel 
Lippmann received the Nobel Prize in phys- 
ics for color photography! Furthermore, 
he obtained true colors without any dyes. 
Lippmann was an important physicist and 
inventor, and he held positions as Professor 
of Experimental and Theoretical Physics at 
the Faculty of Sciences Laboratory in Paris 
and Director of the Research Laboratory 
(later part of the Sorbonne). He was a giant 
in his time but is relatively unknown today. 

So what did Lippmann do to impress 
the Nobel committee? He proposed and 
demonstrated a way to encode color infor- 
mation into photographic plates so that 
colored images could be observed through 
the selective reflection of light. This is the 
same effect that gives rise to the iridescent 
color of hummingbirds through the diffrac- 
tion of light. Diffraction was explained in 
Appendix B and illustrated in Figure B.4. 
Lippmann’s idea was sophisticated and sim- 
ple and almost impossible to realize with the 
materials available to him. He proposed an 
arrangement consisting of a plate with a thin, 
fine grained panchromatic photographic 
emulsion in contact with a mercury mirror 
as illustrated in Figure G.4. Light would be 
directed through the plate and emulsion, 
and would be reflected back by the mercury 
surface. The phase of the reflected light is 


shifted by 180°, and the incoming and outgo- 
ing beams interfere to give nodes and loops 
(bright spots) separated by the wavelength 
of the light divided by two. If we take red 
light from a He-Ne laser with 1 = 633 nm as 
an example, we immediately see Lippmann’s 
dilemma. The separation of the layers (lam- 
ina) in the latent image is 316 nm, and can 
only be recorded with extremely fine grain 
film. If blue light is added, a set of layers with 
smaller separation is produced; and so on as 
other colors are added. 

Lippmann devoted considerable time to 
producing fine-grain, high-resolution film 
and optimizing developer solutions, and he 
was able to obtain color images in spite of 
the absence of true panchromatic film. This 
feat can now be reproduced in an under- 
graduate laboratory by using commercial 
holographic plates with a resolution of 
more than 1000 lines/mm. 

Lippmann’s method of interference color 
photography was not practical for photog- 
raphers, but his ideas have found new life 
in laser-based optical technology. Dennis 
Gabor was fascinated by Lippmann’s ideas 
and set about making interference patterns 
in 3D space. The result was the hologram, 
another Nobel Prize-winning invention. 
Reflection-type holograms can be made by 
exposing an emulsion to laser light scattered 


Emulsion 


Mercury 


FIGURE G.4. Lippmann color photography illustrated with a single color of light. 
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from an object and simultaneously to light 
directly from the laser at a different angle. 
The reflecting planes in the developed film 
are similar to Lippmann’s color photographs, 
and the reflection hologram is known as 
a Lippmann hologram. Also, Lippmann- 
Bragg volume holograms are used as filters 
in optical instruments, and the concept has 
been exploited for storing information in 
standing-wave memory devices where lay- 
ers of information are recorded at different 
depths in an optically sensitive material. 
Gabriel Lippmann provides another exam- 
ple of an inventor who was ahead of his 
time. Fortunately, Lippmann was acknowl- 
edged and celebrated during his lifetime for 
a number of contributions. 


Modern color film: Color film as we know 
it is based on the subtractive process. In 
the developed film light is filtered through 
yellow, cyan, and magenta layers before it 
reaches the eye, as illustrated in Figure G.3. 
A surprisingly complete description of this 
method with the correct order of sensi- 
tized layers and the concept of coupler dyes 
was described by Rudolph Fischer in 1911. 
Fischer was prescient, but the time was not 
right for the realization of his dream. His 
sensitizers and dyes diffused between lay- 
ers, and he could not obtain satisfactory 
color photographs. It would be almost 30 
years before the technical problems could be 
overcome, and transparency film could be 
manufactured with dyes that would stay put. 

The magnificent (three-color) Kodachrome 
film for movies and 35 mm stills was brought 
to the market by Kodak in 1936, and was 
not retired until 2009. This product was the 
brainchild of brilliant amateurs. Leopold 
Godowsky, Jr., and Leopold Mannes, who 
shared interest in music and photography, 
met as boys in New York City. They rec- 
ognized the mediocre quality of so-called 


color movies before 1920, and they decided 
to set up a laboratory to experiment with 
color photography. They also worked as 
musicians, a vocation that would continue 
throughout their lives. Godowsky stud- 
ied violin and chemistry and physics at 
UCLA and later mathematics at Columbia. 
Meanwhile, Mannes studied physics and 
musicology at Harvard. 

Fortunately, Godowsky and Mannes 
were able to continue their collaboration by 
mail, and in 1922 they established a real lab- 
oratory in New York. Soon they were on the 
track that would lead to Kodachrome, and at 
the same time they were becoming success- 
ful musicians. Through a chance encounter 
with Lewis Strauss, who was a junior associ- 
ate with Kuhn, Loeb and Co., they were able 
to obtain financial backing. Their work was 
so impressive that in 1930 they were invited 
to move to Rochester to work in the well- 
equipped Kodak Laboratories. The result 
of this collaboration was the development 
of a highly successful color-subtractive 
film that proved to have amazing archival 
properties, and their coworkers quipped 
that Kodachrome was created by “God and 
Man” In 2005 Godowsky and Mannes were 
inducted into the National Inventors Hall of 
Fame. 
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APPENDIX H 


Microelectronics and the 
Path to Digital Sensors 


detailed timeline of developments 
A= photography technology can be 

found in the Focal Encyclopedia 
of Photography, 4th Edition. Here I discuss 
a few of the more spectacular advances. 
It is difficult to pinpoint the beginning of 
“electronic photography.’ Photographs 
were “scanned” and transmitted over 
wire in the 19th century, and there were 
digitized images (Bartlane System) and 
radiophotograph fax machines in the 1920s. 
Xerography appeared in 1938, and Russell 
Kirsch scanned a photograph of his baby 
son into a computer in 1957. (Life honored 
Kirsch’s image in 2003 as one of the “100 
photographs that changed the world”) 

The most important advance for digital 
photography, however, was the invention of 
the transistor in 1948 (Bardeen, Brattain, 
and Shockley, Nobel Prize, 1956). This sin- 
gle step replaced the vacuum tube with a 
solid-state device and initiated a revolu- 
tion in electronics. A simple version of the 


vacuum tube (triode) contained a heated 
filament, a cathode, a grid, and an anode. It 
looked a bit like a light bulb, and its func- 
tion was to control the current between the 
cathode and the anode with a small voltage 
on the grid. It acted like a valve or gate. The 
transistor, its replacement, also controls the 
flow of current by means of a small voltage 
(or current), but it is smaller, faster, more 
reliable, and uses less energy. The transistor 
was probably the most important invention 
of the 20th century. 

The next major step in the quest for 
smaller, more efficient electrical circuits 
was the design of the integrated circuit in 
1958 by Jack Kilby (Nobel Prize, 2000) and 
Robert Noyce. Kilby was a new employee 
with Texas Instruments (TI), who was work- 
ing on the problem of dealing with large 
numbers of components in electrical cir- 
cuits. His seminal idea was to put the entire 
circuit in a single piece of semiconductor (a 
chip). The time was right, and he was able to 
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demonstrate a prototype in a few months. 
Kilby remained at TI, and he obtained over 
60 patents, including the patent for the 
electronic calculator. Another integrated 
circuit design was developed independently 
by Noyce at Fairchild Semiconductor a few 
months later. The design was not only differ- 
ent, but it solved some significant problems 
in Kilby’s design. Noyce, who had been a co- 
founder of Fairchild, went on to co-found 
Intel Corporation. He passed away too soon 
to share the Nobel Prize with Kilby. 

With the development of the transistor 
and integrated circuits, the race for higher 
and higher densities of circuit components 
was on. This is one of the great success 
stories of technology in the 20th and 21st 
centuries. In 1965, Gordon Moore, another 
founder of Intel, noted that the number of 
transistors that could be put into an inte- 
grated circuit was increasing exponentially 
with time. In fact, the number of circuit ele- 
ments has continued to double every couple 
of years from 1971 through at least 2004; 
and other measures of the microelectronic 
revolution such as hard disk storage capac- 
ity, RAM capacity of computers, or pixels 
per unit area in sensors were also increasing 
exponentially. This trend is now referred to 
as Moore's Law. 

In the 1960s the stage was set for the 
development of image sensors. The break- 
through occurred in 1969, when George 
E. Smith and Willard S. Boyle at AT&T 
Bell Labs invented charge-coupled devices 
(CCDs). These were initially just mem- 
ory devices with shift registers that could 
shift charges from cell to cell without loss. 
However, it was soon realized that the 
charges could originate via the photoelec- 
tric effect in photo cells. Images were being 
captured with CCD image sensors in 1970, 
and digital photography was born. Smith 
and Boyle have received numerous awards 


for their invention, and they were both 
elected members of the National Academy 
of Engineering. In addition to recognition 
by the microelectronics community, they 
were honored with the Progress Metal of 
the Photographic Society of America, and 
in 2009 they shared the Nobel Prize in 
physics. The other half of the award went 
to Charles K. Kao for the development 
of ultrapure fiber optic cables that have 
revolutionized communications. 

This is not quite the end of the story 
for image sensors, since it is evident that 
complementary metal-oxide semiconduc- 
tor (CMOS) devices are appearing in many 
high-end cameras in place of CCDs. CMOS 
detectors are active-pixel sensors that con- 
tain an array of pixels, each of which has 
transistors and a readout circuit in addition 
to a photodetector. (These descended from 
primitive MOS sensors from the 1960s and 
early active pixel MOS sensors invented 
by Peter J. W. Nobel (1968) and Savvas G. 
Chamberlain (1969).) These sensors are 
easily fabricated by standard semiconduc- 
tor techniques and are, therefore, cheaper 
than CCDs, but initially they were viewed as 
second-rate sensors. However, research at 
the Jet Propulsion Laboratory in the 1990s 
and at camera companies such as Canon 
since that time has changed the picture. 
CMOS sensors are now recognized as 
competitive with CCDs in image quality in 
addition to offering advantages in cost and 
energy requirements. 

The culmination of this story for our 
purposes is, of course, the invention of the 
digital camera. This achievement is cred- 
ited to Steven J. Sasson. Sasson was a young 
electrical engineer working for Eastman 
Kodak Company in 1975. His assignment 
was to determine whether a camera could 
be built using solid-state electronics instead 
of film. Here the term camera implies image 
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capture and recording ina portable unit. The 
result was an 8lb, 0.01 megapixel prototype 
which was patented in 1978 by Sasson and 
his then-supervisor Gareth Lloyd. It was 
only in 2009, after digital imaging became 


A 


a multi-billion-dollar industry, that Sasson 
started receiving adequate recognition. He 
has received an “Innovation Award” from 
the Economist magazine and an honorary 
degree from the University of Rochester. 
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APPENDIX | 


Irradiance and 
luminance Units 


lux is the measure of how much of 
Ecas is flowing through unit 

area in unit time. Energy per square 
meter per second is an example. This can 
be measured in units of joules/meter’sec or 
J/(m?s). Since J/s is defined as a watt (W), 
we can write the flux as W/m”. So far I have 
been describing the intensity of light, which 
is the same as irradiance. That is great for 
a physicist, but for engineers dealing with 
illumination, it is not adequate. 

The simplest way to proceed is to note 
that illumination engineers measure illumi- 
nance in units of lumens per square meter or 
ImIm?, and this flux is defined as a lux (Ix).' 
Therefore, the lumen, under certain condi- 
tions, is proportional to energy per second, 
and it looks a bit like a watt. The difference 
between watts and lumens is that a watt 
of green light appears to be much brighter 
than a watt of red light, while lumens have 
been designed so that a lumen of any color 
in the visible region appears to be equally 


bright. This sounds like psychophysics, and 
it is. We have to ask human observers to 
match the brightness of different colors of 
light, or to measure by some means the sen- 
sitivity of the average human eye to light of 
different frequencies. This has, in fact, been 
done for light-adapted (photopic region) 
and dark-adapted (mesopic region) human 
eyes, and the associated luminosity func- 
tions are shown in Figure 1.1. 

The standard lm is approximately 1/683 
W at 555 nm and values for the other 
wavelengths are based on the photopic 
luminosity curve. For example, at 400 nm 
1 W is about 0.27 Im and at 700 nm 1 W is 
about 2.8 lumens. 

Suppose a light beam containing a variety 
of colors (wavelengths) is viewed by a human 
observer. Furthermore, let the spectral power 
density be given by the function J(1) in watts/ 
(unit wavelength). The apparent brightness 
of this beam can be computed if the standard 
luminosity function f (A) is known: 
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Dark-Adapted 
Eye 


Luminosity Function 


400 450 500 


Light-Adapted Eye 


550 
à (nm) 
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FIGURE l.1. Luminosity functions for the human eye: dark-adapted (blue curve) and light-adapted 


(green curve). 


Flux(lumens) = 683 = fp AYA)JdA 


where the range of integration is 400 nm 
to 700 nm. This just means that each incre- 
ment of the power curve has to be adjusted 
by the luminosity curve before it is added to 
obtain the total number of lumens. 

This comparison of units is necessary for 
understanding the characteristic curves for 
film and sensors. However, I have not given 
the standard definition of a lumen. This 
requires that we consider a localized light 
source with light emitted in all directions. 
So here is the definition: 


A light bulb that produces 1 candela iso- 
tropically emits one lumen/steradian in all 
directions. Therefore: 1 candela = 1 Im/sr. 
(The total solid angle for all space is 411 ster- 
adians, and at a radius of 1 ma steradian (sr) 
is the solid angle that cuts off 1 square meter 
on the surface of the proscribed sphere.) 
Also, we have the foot-candle (fc), defined 
as fc = Im/ft? just as lx = lIm/m?. 


Notes 


1. Lux: http://en.wikipedia.org/wiki/Lux 


APPENDIX J 


All About Depth-of-Field 
and Hyperfocal Distance 


he objectives of this appendix are 
the following: 


1. To explain the connection between 
depth-of-field (DoF) and the resolu- 
tion in an image. 


2. To derive equations for front and back 
depths-of-field. 


3. To introduce the concept of the hyper- 
focal distance. 


The variables required for the DoF com- 
putation are illustrated in Figure J.1. For a 
lens of focal length f, an object at a distance 
p from the lens will be focused at a distance 
q behind the lens. For thick or multi-com- 
ponent lenses, p is measured from the first 
principal plane and q is measured from the 
second principal plane. The plane perpen- 
dicular to the optical axis at distance p is 
called the focal plane. Here I assume that p 
and q are much larger than the separation 
between the principal planes. 


It is clear that an object located a dis- 
tance D, in front of the focal plane or D, 
behind it will not be in perfect focus. In fact, 
the defocused rays will spread into a cone 
that will project a circular cross-section on 
the focal plane. If the resulting circular spot 
on the focal plane is sufficiently small, we 
say that the off-plane objects are within the 
depth-of-field for the distance p, the aper- 
ture 6, and the focal length f. The maximum 
acceptable spot is called the circle of confu- 
sion c in the focal plane, and the associated 
distances D, and D, are back and front 
depths-of-field and their sum is simply the 
DoF.! 

The usual starting point is to determine 
the acceptable circle of cand to compute the 
DoF from it. One way to proceed is to set 
the angle 0 based on the resolving power of 
the human eye. 20/20 vision suggests that 0 
can be as large as 1 deg/60 or 1 arcmin, and 
the diameter of c can be calculated with the 
relation c = p- 0. However, this choice would 
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FIGURE J.1. Depth-of-field diagram. 


only be appropriate if images were always 
viewed from the proper prospective point, i.e. 
the point where the field of view equals that 
of the camera at the time of the exposure. In 
this formulation the depth-of-field depends 
only on the distance p, aperture 6, the chosen 
angle @. This is reasonable only if all wide- 
angle prints, other than murals, are viewed 
close to the eye, and telephoto prints are 
viewed from a distance. Of course, the viewer 
usually does not know the proper prospective 
point and tends to view all hand-held prints 
from about 13 in. In slide presentations the 
audience sits at a fixed distance from the pro- 
jection screen even though the focal lengths 
used to make the images may have varied 
over a wide range. The relationship between 
image size and viewing point or prospective 
is illustrated in Figure 4.2. 

It is more reasonable to define a constant 
circle of confusion (CoC) in the image." In 
Figure J.1 the CoC is denoted by c’ on the 
sensor. CoC or c’ is related to c in the focal 
plane by the equation c’ = (q/p) - c where 
(q/p) is the magnification m of the lens sys- 
tem. In film photography the CoC was often 


chosen to be the image width S divided by 
1500. This value makes sense for digital 
photography also for small prints, but for 
prints larger than 11 in. x 14 in. the divi- 
sor should probably be increased to 3000. 
Once the CoC has been chosen, c in the 
focal plane is uniquely determined, and we 
can proceed with the derivation of the DoF. 

From Figure J.1 and using the concept of 
similar triangles we see that c/6 = D,/L, = 
D,/L,and L, =p + D, and L, = p-D,. From 
these simple relations we obtain: 

E D c 


D 
= 1 and = 2 
ô p+D, ô p-D 


(J.1) 


2 


The equations in Equation (J.1) can be 
solved for the front (near) depth-of-field D, 
and the back (distant) depth-of-field D, to 
obtain: 


D=“P and D, = LP 
c 


ô+c (4.2) 


It is interesting to note that if the focal 
plane distance p is adjusted so that 6 = c, 
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the back depth-of-field becomes infinite and 
the front depth-of-field is just p/2. This special 
case is discussed below. Here I will assume that 
6 » c so that c can be neglected in (Equation 
J.2), and a simple approximated equation can 
be derived for the DoF. Also, I assume that 
q = f, replace c with CoC - (p/q), and use the 
definitions m = q/p and N = f/6 to obtain: 
DoF = D, +D, a (J.3) 


This equation can be written in different 
ways to explicitly show the dependence on 
fand 6, but dependences on the F-number, 
N, and the magnification, m, are easy to 
remember and to use. 

In close-up and macrophotography q 
becomes significantly larger than f. In fact, 
for a simple lens described by the conjugate 
equation 1/p + 1/q = 1/f, we find that IN 
= q/d = N- (1 + m). Therefore, it becomes 
appropriate to express the depth-of-field as: 


2-N ,¿CoC 


DoF = = (J.4) 


In the regime where neither c nor ô can 
be neglected, D, and D, are not equal and 
the equations in (J.2) must be used to com- 
pute the front and back depth-of-fields. 
As previously noted, when 6 = c the back 
depth-of-field extends to infinity, and the 
front depth-of-field reaches halfway back to 
the camera aperture. All of this happens for 
a specific value of the focus distance p, and 
following Kingslake we denote this value as 
h, the hyperfocal distance.’ From Figure J.1 
and similar triangles, we see that 0-h = c 
and also that 9. q = CoC. Therefore, we can 
write 


0 OG 2 P 
4 0 CoC N.CoC (J.5) 


where, in the last step, g has been approxi- 
mated by the focal length f. 

With cheap fixed focus cameras from 
film days it made sense to set the focus at 
the hyperfocal distance. This also worked 
well for manual focus lenses in emergency 
situations such as battlefield conditions. 
However, with high-resolution lenses and 
sensors in modern digital cameras the 
hyperfocal concept is less useful. Those who 
make large prints or view images on large 
monitors know that CoC must be adjusted 
downward from the value found in tables 
or on most online calculators. As discussed 
above, in demanding work the CoC should 
probably be no larger than the sensor 
dimension divided by 3000. For a FF cam- 
era this amounts to 0.015 mm. Consider, for 
example, a 24 mm f/2.8 lens that is used for 
night sky photography. The hyperfocal dis- 
tance from (Equation J.5) is 28 m or 94 ft. 
The infinity markings on lenses are seldom 
accurate enough for night sky photography, 
and the proper setting must be determined 
manually with a 10x setting in Live View 
mode. It makes little sense to try to set 94 ft. 
rather than infinity, and for best resolution 
it is best to focus on the object of interest 
rather than trying to set the hyperfocal dis- 
tance. Furthermore, when focus is set at 
infinity, the closest acceptable resolution 
occurs at the hyperfocal distance. 

My own experience with a 14 mm lens at 
f/2.8 agrees with this idea. When I focused 
at 8 ft. and then refocused at infinity, the 
improvement in the sharpness of distant 
objects was evident; and the difference was 
more apparent when the lens was used on 
an APS-C camera with higher pixel den- 
sity. At f/16 the difference in sharpness was 
minimal for full frame cameras. However, I 
found that light filtering through the foliage 
of trees was particularly sensitive to differ- 
ences in resolution. Nearby objects were 
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always acceptably sharp. In closing, I note 
that others have cast doubt on the useful- 
ness of the hyperfocal distance for digital 
photography.?* 


Notes 


1. Kingslake, R. (1951). Lenses in 
Photography. Rochester, NY: Case-Hoyt, 
Corp., chap. 5. 


2. Koren, N., Depth of Field and Diffraction: 
http://www.normankoren.com/ 
Tutorials/MTF6.html 

3. Merklinger, H. M., Depth of Field 
Revisited: http://www.trenholm.org/ 
hmmerk/DOFR.html 
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86-7 

diffractive optics (DO) 50 

digital cameras 109, 127n, 187, 262a; 
design of 18-19; and field of view 24; 
single lens reflex see DSLR cameras 

digital photography, color spaces for 155-7 

digital sensors 167-8; path to 26la—3a 

digitization 176 

diopters 95-6, 108n, 132, 243a; HOYA 
coated 96 

dipole radiator, radiation pattern for 118 

dispersion 14, 47 

distance: clues for 212; perception and size 
219-20 

distortion 21, 47; and fisheye lenses 70 

distractions 216 

dorsal stream 144 

double opponency cells 164 

downstream noise 177 

Driffield, Vero Charles 173 

Drude-Lorentz theory 55n 

DSLR cameras 16, 18-19, 64, 97, 108, 195; 
retrofocus lenses in 64; sensors in 74, 
178; wireless control in 107 

Duboscq, Louis Jules 231a 

DxO Optics Pro, RAW conversion 183 

dynamic range: determination of 175-8; 
fixed 226; versus ISO amplification 178 

dyschromatopsia 144 


E 


Eastman Kodak Company see Kodak 

edges: detection of 164—5, 224; 
enhancement of 199 

Edison, Thomas 231a 

Efke IR film 124 
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Einstein, Albert 9-10 

electric currents 119 

electric fields, transverse (TE) 119-20 

electromagnetic (EM) waves 9, 115 

electrons 167 

emission spectrum, CRT and LCD types 
251a 

Empedocles 6 

emulsion, panchromatic 127n 

energy: light as 5; source of 5-6 

enlargers 25; historical note on 231a-2a 

entrance pupil 57, 61-3 

equidistant projection fisheye 71, 72 

equiluminance 218 

equirectangular projection 80, 81 

equisolid-angle projection fisheye 72 

equivalence, achievement of 83-4, 88-9 

equivalent F-number see scaled F-number 

equivalent focal length 88 

equivalent image 88 

Euclid 6 

experiences, representations and 
reminders of 214 

exposing to the right (ETTR) 180, 192 

exposure 11, 14, 87-8; meter 181—2; for 
pinhole cameras 29; and plate density 
173; settings for high image quality 
178-82; and signal clipping 174; time 
and filters 111 

extenders, and lenses 100-3 

extensible metadata platform (xmp) file 
183-4 

extension 93, 97-8; bellows 97; and 
magnification (m) 97; tubes 94, 97-8, 
102-3, 243a 

extrapolation, mental 224 

eyes 224; age-related changes in 250a; and 
chromaticity 154; and color illusion 4; 
and color perception 246a—7a; colors 
invisible to 139; entrance pupil of 57, 
61-3; luminosity functions in 266a; 
movement of 224; MTF of 135-7; 
saccadic movements 213; sensitivity of 
137-40; simplified schematic of 132; 
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stimuli and responses 136; structure of 

131-3; versus cameras 245a—6a; and 

vision 131; visual acuity of 133, 135-7 
Eyescan 76 


F 


F-number (f/N) 93-4, 99, 191; for common 
sensors 87; definition 14; equivalent 88; 
macrophotography optimum 105-6 

face recognition 13 

face or vase? illusion 213 

Fairchild semiconductor 262a 

false periodic patterns 185 

Feltus, Alan 228 

Fermat, Pierre de: principle of least time 
32-4, 65, 236a 

Feynman, Richard P. 34, 35n, 233a; 
sum-over-path quantum theory 234a 

fiber optic cables 262a 

field curvature 47 

field of view: of eye 137; for simple lenses 
60 

film 15-18, 167-8; capturing photons 
with 253a—9a; color 15, 258a; color 
reversal 255a; discontinued 19n—20n; 
IR sensitive 124; modern color 258a; 
monochromatic (black and white) 173, 
254a; panchromatic 255a; Polaroid 161; 
rolls 15; silver sensitized 3; speed of see 
film speed; tungsten 127n 

film speed 17; definition of 172-5 

filters 127n; absorption 110-15, 114; 
anti-aliasing (AA) 185, 197; blocking 
126; color 109-10, 114—15; definition 
of 109-10; and exposure time 111; 
graduated 112; high-pass 199, 200; 
interference (dichroic) 110; need for 
109-29; neutral density (ND) 109-14, 
111; polarization 109-10, 115-17; 
red and infrared 124; stacking of 117; 
sunset 114—15; transmission 123-4; 
visible light blocking 123—4; warming 
114; Wratten 127n 


first-order theory 46 

Fischer, Rudolph 258a 

fisheye view 73 

Fisheye-Hemi 75 

fixation 213 

fixer 255a 

Fizeau, Armand 8 

flare 54 

flash: off-camera extension and mounting 
bracket 104; supplemental 103, 104 

flint glass 47 

Florence, Hercules 3 

flow, impressions of 218, 219 

fluorite 50 

flux 265a—6a 

focal length (f) 13, 95; definition of 13; 
equivalent 88; and magnification 92; 
and perspective 21, 22; variable 58; 
wavelength dependent 66 

FocalBlade 200 

focus 93-4, 191; automated 58; breathing 
203; fixed point of 226; point of 227 

Focus Magic 199 

focusing rail 107 

foot-candle 266a 

Foucault, Jean Bernard Léon 8 

Fourier frequency 201 

Fourier series analysis, of square wave 
image 197 

Fourier transform 204n 

fovea centralis 133, 135, 137-8, 171, 211 

Foveon X3 technology 170 

Fraunhofer, Joseph von 7, 52 

Fraunhofer lines 49 

frequency, and contrast 201 

Fresnel, Augustine-Jean 7, 51 

Fresnel lenses 50 

Fuji GX617 77-8 

Fujicolor PROVIA 100 reversal film 253a, 
255a 

Fujifilm 16; panoramic cameras 76; Provia 
253a, 255a; Super CCD sensors 168; 
X-Trans CFA 172 

full-well capacity (FWC) 169, 174, 176 


Fumane Cave, Italy 209 

functional magnetic resonance imaging 
(fMRI) 143, 210 

fusiform gyrus 210 

fusion 113-14 


G 


Gabor, Dennis 257a 
Galileo 37, 98 
gamma correction 185-6 
gamma stretch 185 
gamut, of RBG color space 153-4 
ganglion cells 132, 134, 143, 162; 
in retina 163, 165 
Gauss, Carl Friedrich 45, 60 
Gaussian function 191 
Gaussian optics 60—1, 241la—2a 
Gaussian probability 204n 
Gemma-Frisius, Reinerus 2 
gene recombination, unequal homologous 
249a 
genetics, of color vision anomaly 248a—50a 
ghosting 54 
GigaPan 80; tripod heads 77 
glasses, standard (Schott) catalog of 49 
Godowsky, Leopold, Jr. 258a 
Goethe, Johann Wolfgang von 158-9 
Goethe-Hering theory 158-60 
Google smartphones 192 
grain/graininess 15, 175, 196 
Granger, Edward 201-2 
gratings 200-1 
gray, concept of 145 
Gregory, James 7 
Grimaldi, Francesco 7 
Grotte Chauvette, France 209 


H 


Harrison, Karl 81 
Heisenberg, Werner 55n 
HeliconFocus 106 
Heliopan 124 
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Helmholtz, Herman von 148 

herapathite 128n 

Hering, Ewald 146, 159 

Hering theory 159-60, 164 

Hermann grid illusion 163 

Hero (Heron) of Alexandria 6, 32; law of 
reflection 32, 234a, 236a 

Herschel, Sir John 3, 231a 

Hertz, Heinrich 45 

hiatus 59 

high dynamic range (HDR) 106; Expose 
113; processing 113; technique 178; tone 
mapping 113 

high image quality, exposure settings for 
178-82 

highlights, computer-adjusted 112 

Hill’s fisheye lens 74 

holography 257a—8a 

horizontal field of view (HFOV) 72, 74 

hot spots, in IR photography 124 

Hoya 124; R72 IR filter 125 

Hubel, David 163, 165 

Hubel and Wiesel experiment 163 

hue 148 

hue, saturation, brightness (hsb) system 
160-1 

human brain see brain 

human eyes see eyes 

human photo pigments see photo 
pigments 

human vision 4, 122-3; limits of 131-41; 
and resolution 200 

Hurter, Ferdinand 173 

Huygens, Christiaan 7, 33 

hyperfocal distance 267a—70a 


| 


ideal fisheye see equidistant projection 
fisheye 

Ilfochrome 256a 

illuminance 173; units 265a—6a 

illumination: of detectors 64; of the image 
61-3 
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illusions: of motion 218; unrecognized 218; 
visual see visual illusions 

illustrators 252a 

image capture 186-7; discovery of 1 

image clarity see image quality (1Q) 

image quality (IQ) 89, 189-90; and sensor 
size 18; and shutter speed 179-80 

image sensors 262a; capturing photons 
with 168; CCD and CMOS 168-9, 
see also sensors 

images: attraction of 214—17; belief in 4; 
black and white 161; blurring of 190-1, 
190; capture of see image capture; 
circles and sensor sizes 75; color and 
color monochrome 161; combination 
of 227; computer manipulated 3; 
convolution of 199; creation with 
simple lens 59; curvature of 44, 48; 
determinants of appearance 84—8; with 
different focal points 107; duotone 161; 
enhancements 113; equirectangular 
81; equivalent 83-9; exposure settings 
for high quality 178-82; fisheye 75-6; 
formed by compound lenses 59-60; 
framed or boxed 226; grainy 15; hidden 
218; illumination of 61-3, 62; infrared 
(IR) 124-6, 126; JPEG (JPG) 182; latent 
253-4a; learning from 215-17; low 
dynamic range (LDR) 113; manipulation 
of 183-4, 208; monochrome and 
monotone 160-—1; necessary quality of 
131-41; negative 3; for panoramas 79; 
perceived quality of 199; perception 
of 211-14; and perspective 21-5; 
photographic limitations to 226-7; 
processing of 167-88, 199-200; 
quality of see image quality (IQ); real 
93; repulsiveness of 214—17; saving 
of 182-3; scioptic lens projection 70; 
shock value of 219; softness of 190-1; 
stabilization of 58, 191; storage of 
182-3; successful 215-17; ultra violet 
(UV) 123-4, 124; virtual 92-3; 
wide-angle 112 


imagination 221 

impact 208 

impressions, of flow 218, 219 

incidence 119; angle of 32 

inference 213 

inquisitive tendencies 214—15 

integrated circuits 261-2a 

Integrated Color Corporation 158 

Intel Corporation 262a 

intensity 110; loss of 63 

interference 52; color photography 257a; 
constructive and destructive 8; effects 
7 

International Commission on Illumination 
(CIE) 246a 

International Organization for 
Standardization (ISO) 172 

inverse problem 211 

iPads 81 

iPhones 81 

iris, of eye 132 

irradiance units 265a—6a 

ISO amplification, versus dynamic range 
178 

ISO invariant sensors 177 

ISO number 17 

ISO setting 177 


J 


Jackson, William Henry 12 

Jenoptik CoastalOpt 60 mm 123 

Jet Propulsion Laboratory 262a 

Johnson-Laird, Philip 228 

Joint Photographic Experts Group (JPEG) 
182 

Jonathan I. 145 

judges 208 

Juza photo site 99 


K 


Kandel, Eric 225-6 
Kao, Charles K. 262a 


Kenko: 1.4x teleconverter 243a; extension 
tubes 97 

Kepler, Johannes, theory of vision 37 

Kilby, Jack 261-2a 

Kingslake, Rudolf 21, 48, 52, 269a 

Kirk, mounting brackets 105 

Kirsch, Russell 261a 

Klimt, Gustav, Adele Bloch-Bauer 228 

Koch, Christof 224 

Kodachrome: color to monochrome 
conversion 114; film 258a; subtraction- 
process film 256a 

Kodak 15, 124, 127n, 171, 199, 201, 258a, 
262a; CFAs 171, 172; discontinued film 
19n—20n; Wratten 18A 123 

Kolor PanoGear 80 

Kramers, Hans 55n 


L 


Lambert, Johann 127—8n 

Land, Edwin H. 115, 128n, 161 

Land theory 161-2 

Lascaux, France 210 

lasers, gas 52 

lateral geniculate nucleus (LGN) 143 

Lawrence, George R. 12 

Leica 19; M8 155, 156 

lens design theory 13-14 

lens maker’s equation 40, 55n; derivation 
of 239a—40a 

lens shift 25 

lenses 11, 13-15; achromatic 47, 48; 
apochromats 48; aspheric 47; auto- 
focus 54; Barlow 99; biconvex 44—5; 
close-up 94, 95-6; combinations and 
magnification 244a; compound 40-1, 
47, 59-60, 93; concave 43; conventional 
72; convex-concave (meniscus-shaped) 
44—5; design tradeoffs 46; development 
of 41n-2n; diffraction limited 194, 202; 
effective aperture 62; entrance pupil 
of 57, 61-3, 63; evaluation criteria for 
45; and extenders 100-3; fisheye 25, 
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70-6, 73-4, 82, 98; fixed 100; fixed 
focal length 25; Fresnel 50; glass 37; 
hemispheric 73; of human eye 131; 
interchangeable 98-9; long 92; macro 
91, 94, 101, 101, 243a—4a; making for 
photography 43-55; micro 186; MTFs 
for 194, 196; narrow field 24; need 
for 27—9; negative 43, 47; no-parallax 
point (NPP) 77; normal 23; ox-eye 70; 
pair with exaggerated thickness 41; 
performance of 193-6; positive 43, 
47; prime 25, 95-6, 96, 99; quartz/ 
fluorite 123; real 57-67; rectilinear 
23, 72; reflections from 51; retrofocus 
64; scioptic 70-1; shift and tilt 23, 25; 
simple 34, 58, 59-60, 98-9, 239a; simple 
convex-convex spherical 39; simple 
spherical 44; single thin 92; speed 14; 
super-zoom 203; superachromat 48; 
supplementary 94, 95-7, 96, 101-2; 
symmetrical 94; teleconverter 99; 
telephoto 24, 50, 57, 98-9; thick 59-60; 
thin 37-42, 58, 92; wide-angle 23-4, 
69-70; working of 31-5; zoom 24-5, see 
also metalenses 

Life of Pi, The 4 

Life Pixel 123, 126 

lifesize 91 

light: cone of 62; definition of 5-10; 
electric field of 118; as electromagnetic 
(EM) radiation 9, 115, 233a; mixing 
148-9; polarized 115; quanta 9; red 
138, 234a; reflection of 120; refraction 
of 120; source of 5-6; speed of 8, 31, 
34, 35n; supplemental sources 103, 
104; travelling in straight line 31-2; 
unpolarized 115, 116; writing with 1, 3, 
5, 167 

light emitting diodes (LEDs) 148 

light field, capture of 186-7 

light rays: from object to detector 85; 
geometry of 70 

Lightroom see see Adobe Lightroom 

Lincoln, President Abraham 11 
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line spread function (LSF) 135, 194, 204n 

linear perspective 78, 212 

linear polarizers 115, 116, 128n 

linear scaled fisheye see equidistant 
projection fisheye 

Lingelbach, Elke, Hermann grid illusion 163 

Linhof panoramic cameras 76 

Lipperhey, Hans 37 

Lippmann, Gabriel 257a-8a; color 
photography 257a; hologram 258a 

Lippmann-Bragg volume hologram 258a 

liquid crystal display (LCD): emission 
spectrum 251a; screens 19, 125, 155-6, 
185, 250a 

lithium niobate 185 

live view 108 

Lloyd, Gareth 263a 

longitudinal spherical aberration (LSA) 46 

Lorentz, Hendrik 54n—5n 

low dynamic range (LDR) 113 

LR/Enfuse 114 

Lucretius 6 

luminance 224, 225; cells in retina 162, 
163; equal 218; fluctuations 192 

luminosity, and brightness 265a—6a 

Lytro, Inc. 186, 227 


M 


MacAdam ellipses 154, 166n 

Macleod, Angus 52 

macro couplers 96, 97 

macro lenses 91, 94, 101, 101, 243a-4a 

macrophotography, real world 100-3 

macula region 133; schematic of 134 

Madler, Johann Heinrich von 3 

magnetic fields, transverse (TM) 119, 120 

magnification (m): 1X 93; 4X and greater 
96; defined 91, 93; and diopters 95; 
and extension 97; obtaining very high 
91-108; with single thin lens 92; special 
problems 103; and teleconverters 99 

Magritte, René 220 

Man Ray 220 


Manfrotto: focusing rail 107; mounting 
brackets 105 

Mangelsen, Thomas D. 77 

Mannes, Leopold 258a 

matching functions 150-1, 152 

Mathcad 129n 

Matisse, Henri 228 

MaxMax 126 

Maxwell, James Clerk 8-9, 31, 34, 45, 161, 
256a; electromagnetic theory of 118, 
233a; and tristimulus values 148 

Maxwell’s equations 120 

mechanical vignetting see vignetting 

megapixels 16-17 

melanopsin 132 

memory, visual editing in 224 

mental extrapolation 224 

mesopic vision 138—9 

metalenses 65-6; converging power of 66 

metals, reflection by 122 

metameric pairs 150 

microelectronic chips, as sensors 15-16 

microelectronics 261a—3a 

Microsoft HD View 78 

middle temporal area (MT or V5) 144 

Miethe, Adolph 127n 

mind 208-11, 224, see also brain 

mirror ball 80-2, 81 

mirrors: ellipsoidal 34; reflection from 
234a—Sa, 236a 

Miyamoto, Kenro 73; fisheye lens 74 

Mo-Ti 1 

mobile phones 168 

mode dial 180 

modulation transfer function (MTF) 
135-7, 193-5, 198-9, 201-2; for 
diffraction limited lens 194, 202; for 
high quality lens 196 

Moiré fringes 171, 185 

Monet, Claude Oscar, Impression Sunrise 218 

monitors: calibration of 158; EIZO 
ColorEdge 252a; gamut of 157 

monochromatic film, H and D curve of 
173, 174 


monochromatic vision 247a 

monochrome images 160-1; with color or 
hue 160-1; conversion from color 114 

monotone images 160-1 

Moore, Gordon 262a 

Moore’s law 262a 

motion: detection 224; illusions of 218; 
parallax 77, 212 

moving objects 216 

Miiller-Lyer illusion 212, 213 


N 


N value, definition 14 

natural vignetting see vignetting 

nature: lighting for photography 103-5; 
polarization in 117-22 

negatives 3, 11; digital 183 

neuromagic 225 

Newton, Sir Isaac 6-7, 52; color theory of 
146-9, 158-9; color wheel of 147; light 
mixing by 148; optics experiment of 
147, 247a 

Niépce, Nicéphore 2 

NIK HDR Efex Pro 113 

Nik Sharpener Pro 200 

Nikon 19, 50; D1X DSLR sensors 168; OP 
fisheye 71-2; pixel errors 183; RAW 
conversion 183; super-zoom lenses 203 

no-parallax axis 63 

Nobel, Peter J. W. 262a 

Nodal Ninja, tripod heads 77, 78 

Nodal Zone 59 

noise 175, 192; dark 176; downstream 177; 
sources of 176; upstream 177 

Noyce, Robert 261—2a 

Nyquist limit 197-8 

Nyquist theorem 185, 197; illustration of 
198 


O 


object (O) 61-2; of interest 215; light rays 
from 85; placement of 216, 217 
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Olympus: 50 mm lens 96; RAW conversion 
183; sensors 196 

opponency cells, red/cyan double 164 

opponent color pairs 159 

opponent theory 140, 146 

opsin molecules 247a 

optical aberration 14 

optical density (OD) 110, 173 

optical materials 49-50 

optical strength 95 

optical surfaces 38-40 

optical vignetting see vignetting 

optics: elementary 31-5; for light beams 
(rays) 31-2; new 21st century 66-7; 
rules of 233a—7a 

Optics Software for Layout and 
Optimization (OSLO) 45 

orthographic projection (OP) fisheye 71, 
72 

Ostroff, Eugene 231a 


P 


Palmer, George 146 

Panasonic, sensors 196 

PangeaVR 81 

panning devices 80 

panorama tools 75 

panoramas 71, 76-82; cylindrical 77-80; 
examples of 79; high-resolution 78; 
rectilinear 77—80; spherical 80-2, 227; 
stitched 79 

panoramic tripod head 77 

Panoscan 76 

paper: profiles of 158; silver sensitized 3 

parallax, motion 77 

parametric editing 184 

paraxial approximation 39 

paraxial theory 46 

Paris Photographic Society 231a 

particle theory 7 

path length 33-4 

paths, stationary 34 

Pentax 19 
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per cent luminance values 152 

perception, of images 211-14 

periodic sampling, inadequate 184—5 

perspective 84—5; definition of 21—5; and 
focal length (f) 21, 22; linear 23; point 
22, 23; and wide-angled lenses 23-4 

Petzval, Joseph 45, 48 

Petzval sum 48, 73 

pewter, bitumen-coated 2 

Phase Fresnel (PF) 50 

phase shifts 65-6 

Phillips trichroic prism 170 

photo excitation, of silver bromide 254a 

photo pigments, sensitivity of human 248a 

photochemistry 168 

photodiodes 174 

photoelectric sensors 3 

photograph 3; definition of 21; orientation 
and lighting 212 

photographers 252a 

photographic film see film 

photographic plates 11 

photographs: combining of 106; definition 
of good 207-9; panoramic 76-82 

photography 3, 167, 226; close-up 91; color 
see color photography; competitions 
207-8; definition of 1-4; electronic 
261a; film-based 15; image limitations 
226-7; infrared (IR) 109, 122-6; 
interference color 257a; invention 
of 1-2; lighting for nature close-ups 
103-5; macro 91, 107-8; making lenses 
for 43-55; reflections on art in 221; 
telescopic 91; ultra violet (UV) 109, 
122-6; and vision 223-9 

photojournalism 3 

PhotoKit Sharpener 200 

Photomatix Pro 113 

photons 9-10, 17, 34, 133, 140, 167, 
234a; arrival of 174; capture of 168, 
253a—9a; detectors for 168-9; in film 
photography 168; two-slit experiment 
234a 

photophysics 168 


photopic vision 137-8 

photoreceptors: effective number of 137; 
of eye 132 

Photoshop 25, 63, 65, 74, 113, 158, 177, 
184; Channel Mixer feature 114, 
126, 160; color spaces in 156; filter 
simulation 112; focus stacking 106; 
grayscale 160; Info palette 160; and 
JPEG images 182; sharpening filters 200; 
Warp tool 75 

Picasso, Pablo 228 

pigment epithelium 132 

pinhole cameras 27, 28-9; Wood's 72-3 

Pinker, Steven 209 

pixel arrays 169 

pixel pitch 168 

pixels 10, 105, 169, 262a; definition of 16; 
errors 183; of eye 137; and resolution 
89; RGB detectors in 170, 171; 
vignetting 65; width (pitch) 16 

placement 216, 217 

plate density, and exposure 173 

Plato 6 

pleasure, and art 226 

plenoptic cameras 186-7, 227; schematic 
187 

plus diopters 95, 108n 

Poggendorff illusion 212, 213 

point of attention, fixed 227 

point of focus 227 

point spread function (PSF) 135, 190, 
191-2, 198-9, 204n; effect on contrast 
193, 194 

polarization: in rainbows 121-2; and 
reflections 119 

polarized light 115; in nature 117-22; 
resolution of 117 

Polaroid Corporation 115, 128n, 161, 256a 

Polaroid film 161 

positive color method 256a 

PREMO SR camera 12 

presbyopia 37 

presentation: color space and 
gamuts of devices 155; fixed 227; 


three-dimensional (3D) 227; 
two-dimensional (2D) 226 

primary colors 151-3 

primary visual cortex (V1) 143, 224 

principle planes 59, 241a—2a 

printer calibration 158 

prisms: and refraction 147; trichroic 169, 
170 

probability amplitude 234a 

processing: high dynamic range (HDR) 
113, 113; in-camera 182; and sharpening 
199-200 

profile connected space (PCS) 157 

program mode 181 

projection: equirectangular 80, 81; fisheye 
71 

projectors: calibration of 158; three-color 
162 

Prokudin-Gorsky, Sergey 256a 

ProPhoto color space 157 

prosopagnosia 210 

protanomaly 249a 

protanopes 248a, 250a 

protanopic vision, simulation of 249a 

psychophysics 6 

PT-Gui 80 

pupil 135; ratio (p ) 94 

Pythagoras 6, 33 


Q 


quanta 9 

quantum efficiency, of detectors 169, 174 

quantum electrodynamics (QED) 34, 35n, 
233a 

quantum mechanics 55n 

quarter-wave plate 116-17, 129n 


R 


radiation: light as 5; pattern for dipole 
radiator 118 

rainbows, polarization in 121-2 

RAW file format 156, 182-3 


ray tracing 29 

Rayleigh, Lord 28, 52 

rays, description of 60-1 

read noise 175-6 

realism, photographic 228 

reality: re-creation of 223; and sight 223; 
and truth 4 

receptors, visual 133-5 

reciprocity 173-4 

recommended exposure index (REI) 174 

red-green color blindness 144 

reference output media metric (ROMM) 
color space 157 

reflectance, at air/water interface 121 

reflection 52; angle of 32; by metals 122; 
from air to air 237a; from glass sheets 
236a-7a; from mirrors 234a—Sa, 236a; 
Hero’s Law of 32; of light 120; and 
polarization 119; in rainbows 121-2; 
sum-over-paths analysis 236a 

reflectors, hot-mirror IR 125 

refracting surface, spherical 62 

refraction 119; indices of 31, 35n, 119; of 
light 120; and prisms 147; in rainbows 
121-2; Snell’s law of 31-3, 32, 38-9, 46, 
65, 72, 119-20, 234a 

refractive index 40 

relative intensity (1) 185, 186 

relativity, theory of 35n 

rendering intents 157 

resolution 184, 189, 199; and grain size 15; 
and sensors 187 

resolving power, of eye 135 

retina 132-4, 143, 146, 171; color 
opponent cells in 162-5, 247a; ganglion 
cells in 163, 165; luminance (ganglion) 
cells in 163; schematic diagram of 133; 
and standard observer 246a; stimulation 
of 227 

11-cis-retinal 247a 

Retinex theory (Land) 162 

reversing rings 97 

rhodopsin 133 

Rhythm & Hues Studios 4 
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Rochester Camera Co., PREMO SR 12 

rods 132-4, 138-9, 246a; and detectors 
140 

Roemer, Olaf 8 

Rokinson 72 

Rollei film 124 

Ronchi, Vasco 38 

Roundshot VR Drive 80 

Royal Society of London 3 

rule of thirds 216, 217 


5 


saccades 245a 

Sacks, Oliver 145 

Samyang, SP fisheye 72 

Sasson, Steven J. 262—3a 

saturation 148 

scaled F-number 88 

scanning path 211 

scatterers, TiO, 65 

scattering 119 

schemata, personal 213-14 

Schott-Glass 124 

Schwenter, Daniel 70 

scotopic vision 137-8, 247a 

Seidel aberrations see aberrations 

Seidel, Ludwig von 45-6 

Sellmeier equation 54n—5n 

Sellmeier, W. 49 

sensitivity: centers 254a; of eye 137-40; 
threshold 138 

sensitizers 255a 

sensor diagonal (Diag) 105 

sensor speed, definition of 172-5 

sensors 15-18; active-pixel 262a; capturing 
photons with 168; CCD 65, 168-9; 
characteristic curve of 175; circles-of- 
confusion (CoC) for 87; CMOS 64-5, 
124-5, 168-9; digital 167-8, 261a-3a; 
for digital cameras 16; distance to (D) 
94; F-numbers (N) for 87; and image 
circles 75; and image quality 18; ISO 
invariant 177; Kodak spectral response 


155; microelectronic chips 15-16; and 
resolution 187; resolution of 185, 196-8; 
sagittal and meridional test points 195; 
sizes of 17, 75, 88—9; specifications of 
155, see also image sensors 

serine 249a 

sexual stimulation 214 

shadows, computer-adjusted 112 

sharpening: by high-pass filters 200; by 
image processing 199-200; workflow 
for 200 

sharpness 189, 199 

shutter priority 181 

shutter speed 87-8; and blur 111; high 
103-5; and image quality 179-80 

sight, and reality 223 

Sigma 50; 105 mm macro lens 97, 243a, 
243a—4a; super-zoom lenses 203 

Sigma Corporation 170 

signal 175; averaging 192 

signal-to-noise (S/N) ratio (SNR) 17, 
175-7, 192 

silver 3, 253—4a, 256a; bromide 253a, 254a; 
chloride 3, 253a; halides 15, 253-5a; 
iodide 253a 

size, and distance perception 219-20 

Smakula, Alexander 52 

Smith, George E. 262a 

Snell, Willebrord, law of refraction 31-3, 
32, 38-9, 46, 65, 72, 119-20, 234a 

snow scenes 182 

sodium thiosulfate 255a 

softness 190-1 

software: for distortion correction 74; 
image processing 183; Pano2VR 
package 81; processing 113; smart 
sharpening 199; stitching 71; vignetting 
correction 65 

solar flux 123 

solid angles 137 

Solso, Robert 209 

Sony 171; A7R 178; CFAs 171, 172 

spatial frequency 135-6 

spatial resolving power 189 


spectral power distribution 149 

spectral response, of Kodak sensors 155 

spectrophotometers 186 

speed: definition of 172—5; of lens 14; 
ratings 173 

spot metering 182 

square wave image, Fourier series analysis 
of 197 

sRGB color space 156-7 

stacking order 102-3 

StackShot Macro Rail Package 107 

standard colormetic observer 151, 152 

standard human observer 245a—6a 

standard illuminant 166n; D50 154 

standard output sensitivity (SOS) 174-5 

Stanford plenoptic camera 186 

stereographic projection (SP) fisheye 72 

stimuli 149 

stitching, panoramas 80 

Stone Age 210 

stop 61-3; bath 254a 

stops (F-numbers) 14 

Strauss, Lewis 258a 

striate cortex 201 

subject matter, choice of 216 

subjective quality factor (SQF) 201-3 

subtractive process 255a 

subwavelength structure coatings (SWC) 
54 

sun: as source of light and energy 5-6; 
unpolarized light from 118-19 

sunlight, natural 154 

Sunwayfoto, tripod heads 77 

superposition principle 34 

surfaces: air/glass 50; optical power of 39; 
of simple lens 239a 

symmetry, and composition 217 


T 


INDEX S 285 


technical perfection 208 

tele-extenders 94, 243a, see also 
teleconverters 

teleconverters 98-9, 99, 102-3; afocal 100; 
front mounted 100; and magnification 
(m) 99 

telephoto ratio 98 

telescopes 37 

test points 195 

tetrachromacy 250a 

Texas Instruments (TI) 261—2a 

thalamus 146 

thin lens equation 40 

TIF format 183 

Tiffen 124 

tintype 11 

tone mapping, HDR 113 

top-down-awareness 213-14 

Topaz Detail 200 

tourmaline, crystalline 128n 

transistor 261-2a 

transmission 52-3; by color filters 114 

transmittance (T) 110, 110 

trianope 248a 

trichromacy 158-9; color theory 140 

trichromatic vision 247a, 249a; simulation 
249a 

trichromats 250a; color matching in 
250a 

trichromism 146 

triode 261a 

tripods 191 

tristimulus value 148, 150, 152, 154; and 
color management 157-8; ratios 153 

Troy 4 

truth, and reality 4 

tutorial videos 184 


U 


tagged image format files (TIFF) 183 
Talbot, Henry Fox 2-3 

Tamron, super-zoom lenses 203 
Taylor, Dennis 52 


unconscious: and art 225; inductive 
inference 211 

unit focusing 93 

upstream noise 177 
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user error 190 
UV-Nikkor 105 mm 123 


V 


Van Vleck, John 55n 

vanishing point 23 

vase or face? illusion 213 

vectors, sum for two paths 235a 

ventral stream 144 

Vermeer, Johannes 1 

viewfinders 11, 19 

viewing distance 23 

vignetting 63-5, 78; mechanical 64; 
software correction for 65 

virtual reality (VR) 82, 227; cubes 81; 
tripod head 77 

Vishneski, Bob 126 

visibility 189 

vision 224—6; and brain 211; and 
comprehension 145; human see human 
vision; Kepler’s theory of 37; and 
photography 223-9; red/green anomaly 
249a 

visual acuity (VA) 133, 135-7, 141n, 205n 

visual art 208 

visual cortex 246a; and brain 246a, see also 
brain 

visual dissonance 220-1 

visual emissions 42n 

visual illusions 2712-13, 219-20, 221; 
impression of flow 219; shades of gray 
225 

visual receptors, operation of 133-5 

visual system, and artistic advantage 
218-21 

vitreous humor 132 

Vogel, Hermann 127n 


void mode 146 
volition 213 
voltage (V) 185, 186 


W 


wave theory 7-9, 8 

wavelength, versus solar intensity 123 
wavelets 201 

wet plate collodion process 15 

white balance adjustment 114, 126, 145 
White, Larry 203 

Wiesel, Torsten 163, 165 

Wollaston landscape lens 13 
Wollaston, W. H. 44 

Wood, Robert 72 

working distance (w) 94 

Wratten, Frederick 127n 

Wratten system 114 


X 


X-chromosomes 248—50a 
X-Rite 158; Colormunki 158, 186 
xerography 261a 


Y 


Young, Thomas 7, 146, 148, 247a 
Young-Helmholtz color theory 149, 158-9 


Z 


Zeiss, Carl 128n 
Zeiss company 52 
Zeiss formula 86 
Zerene Stacker 106 
zoom 58 


